Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic
Answered

How I can call Crawlera in PERL script?

How i can call Crawlera in PERL script


Best Answer

This should work: 

#!/usr/bin/perl
use LWP::UserAgent;
my $agent = LWP::UserAgent->new();
$agent->proxy(['http', 'https'], "http://APIKEY:@proxy.crawlera.com:8010");
print $agent->get('http://httpbin.scrapinghub.com/ip')->content();

 


Answer

This should work: 

#!/usr/bin/perl
use LWP::UserAgent;
my $agent = LWP::UserAgent->new();
$agent->proxy(['http', 'https'], "http://APIKEY:@proxy.crawlera.com:8010");
print $agent->get('http://httpbin.scrapinghub.com/ip')->content();

 

Hi,


When I run this script from my local XP machine, it works fine but when I do the same from a remote server running on Linux, it says: 500 Can't connect to proxy.crawlera.com:8010 (connect: Connection refused).


Can you tell me what the issue could be? Thanks

It seems some firewall blocking the Crawlera proxy port. Can you please check the firewalls and open the port 8010.



Login to post a comment