Using Crawlera over Wget Request

Posted over 5 years ago by Charlie Yaris

Post a topic
Answered
C
Charlie Yaris

Hi there,


I cannot figure out the correct syntax for using crawlera over wget.


Here's what I tried most recently.


wget -nc -r -l0 -e http_proxy=proxy.crawlera.com:8010 proxy_password=<apikey>: http://httpbin.org/ip WEBSITE


This gave me ERROR 407: Proxy Authentication Required.


I'm not sure if http://httpbin.org/ip is in the right place, or if i've inputted the api correctly. The documentation only includes information about curl, not wget.


https://doc.scrapinghub.com/crawlera.html

0 Votes

nestor

nestor posted over 5 years ago Admin Best Answer

Your apikey is the user and the password is blank


http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password


or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

0 Votes


1 Comments

nestor

nestor posted over 5 years ago Admin Answer

Your apikey is the user and the password is blank


http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password


or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

0 Votes

Login to post a comment