Start a new topic
Answered

Using Crawlera over Wget Request

Hi there,


I cannot figure out the correct syntax for using crawlera over wget.


Here's what I tried most recently.


wget -nc -r -l0 -e http_proxy=proxy.crawlera.com:8010 proxy_password=<apikey>: http://httpbin.org/ip WEBSITE


This gave me ERROR 407: Proxy Authentication Required.


I'm not sure if http://httpbin.org/ip is in the right place, or if i've inputted the api correctly. The documentation only includes information about curl, not wget.


https://doc.scrapinghub.com/crawlera.html


Best Answer

Your apikey is the user and the password is blank


http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password


or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

1 Comment

Answer

Your apikey is the user and the password is blank


http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password


or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

Login to post a comment