Using Crawlera over Wget Request

Posted about 6 years ago by Charlie Yaris

Post a topic

Answered

Charlie Yaris

Hi there,

I cannot figure out the correct syntax for using crawlera over wget.

Here's what I tried most recently.

wget -nc -r -l0 -e http_proxy=proxy.crawlera.com:8010 proxy_password=<apikey>: http://httpbin.org/ip WEBSITE

This gave me ERROR 407: Proxy Authentication Required.

I'm not sure if http://httpbin.org/ip is in the right place, or if i've inputted the api correctly. The documentation only includes information about curl, not wget.

https://doc.scrapinghub.com/crawlera.html

0 Votes

nestor posted about 6 years ago Admin Best Answer

Your apikey is the user and the password is blank

http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password

or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

0 Votes

1 Comments

nestor posted about 6 years ago Admin Answer

Your apikey is the user and the password is blank

http_proxy=proxy.crawlera.com:801

proxy_user = yourapikey

proxy_user = empty or random dummy password

or you can do just: http_proxy=yourapikey:@proxy.crawlera.com:8010

0 Votes