How can we help you today?

Start a new topic
Discussions Scrapy Cloud Technical Help
Answered

Proxymesh with ScrapingHub

Hi,


I am migrating my Scrapy project to ScrapingHub.

I am currently using Proxymesh with Scrapy, therefore I have the following questions:

- Can I use proxymesh with ScrapingHub (Proxymesh requires a fixed IP address from the requests). Does ScrapingHub provide a unique IP address for a project or does it change dynamically every request / day / month?

- If not, how many IP addresses does Crawlera provide?

Best Answer

- Scrapy Cloud IPs are dynamic and can change, we can't provide a specific range. Please see: https://support.scrapinghub.com/support/solutions/articles/22000200422-what-are-the-ip-addresses-used-by-scrapy-cloud-. I would suggest an authentication (username:password) approach to authenticate to Proxymesh, if possible.


- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.

1 Comment
Answer

- Scrapy Cloud IPs are dynamic and can change, we can't provide a specific range. Please see: https://support.scrapinghub.com/support/solutions/articles/22000200422-what-are-the-ip-addresses-used-by-scrapy-cloud-. I would suggest an authentication (username:password) approach to authenticate to Proxymesh, if possible.


- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.

Login to post a comment
More topics in Technical Help
See all 458 topics