Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic
Answered

Crawlera Subscription

I'm new to the scappring world and trying to build a crawler in Python. I'm scrapping data from google search and it blocks my IP after like 30 requests. Thats why I want to subscribe to monthly subscription. My confusions are 


1.Do I really get 150,000 requests in 25$ or there is a catch becaust the offer is too good to be true. Are there any hidden charges?


2. Will I be able to build my custom crawler in python?


3. Will it solve the problem of getting blocked by google?


4. What is the request rate? how many request can I make per second?


Best Answer

Thanks for showing interest in Scrapinghub.


Please find my answers to your queries below:


1. Yes you get to make 150K requests in $25 plan. Only successful requests(with response codes 200, 301/302) are considered in the limit quota.


2. You can build your Crawler and then use Crawlera as the Proxy to pass the requests. Please find the example of using Crawlera with Python requests here .


3. Crawlera can help in evading the bans from Google. There are users that are successfully crawling Google. We currently don't offer a trial period in our Crawlera plans, but we do offer a 7-day money back guarantee if Crawlera doesn't meet your needs. Suggest you go ahead and sign up for our cheapest C10 plan (at $25) and try the product. If it doesn't work the way you expect, contact our support team within the next 7 days and they will process a refund.


4. Crawlera's default request limit is 5 requests per second(rps) for each website. You can refer Requests Limits to know more. 


1 Comment

Answer

Thanks for showing interest in Scrapinghub.


Please find my answers to your queries below:


1. Yes you get to make 150K requests in $25 plan. Only successful requests(with response codes 200, 301/302) are considered in the limit quota.


2. You can build your Crawler and then use Crawlera as the Proxy to pass the requests. Please find the example of using Crawlera with Python requests here .


3. Crawlera can help in evading the bans from Google. There are users that are successfully crawling Google. We currently don't offer a trial period in our Crawlera plans, but we do offer a 7-day money back guarantee if Crawlera doesn't meet your needs. Suggest you go ahead and sign up for our cheapest C10 plan (at $25) and try the product. If it doesn't work the way you expect, contact our support team within the next 7 days and they will process a refund.


4. Crawlera's default request limit is 5 requests per second(rps) for each website. You can refer Requests Limits to know more. 


Login to post a comment