Hello,
I am trying to use proxy to download data from websites.
library(rvest)
library(httr)
library(stringr)
my_proxy <- use_proxy(url="http://example.com",
user_name = "myusername",
password = "mypassword",
auth = "one of basic, digest, digest_ie, gssnegotiate, ntlm, any")
search.page <- "https://scholar.google.ca/scholar?q=Exploiting+Chaos+to+Control+the+Future&hl=en&as_sdt=0,5"
scholar_webpage <- html_session(search.page,my_proxy) %>%
read_html()
However, one error always comes up
Error in curl::curl_fetch_memory(url, handle = handle) :
Received HTTP code 407 from proxy after CONNECT
9. curl::curl_fetch_memory(url, handle = handle)
8. request_fetch.write_memory(req$output, req$url, handle)
7. request_fetch(req$output, req$url, handle)
6. request_perform(req, hu$handle$handle)
5. GET(search.page, my_proxy)
4. eval(lhs, parent, parent)
3. eval(lhs, parent, parent)
2. GET(search.page, my_proxy) %>% read_html()
Does anyone has any suggestions?
Thank you for your time and patience!
Let me know if you need more information.