I am currently using Proxymesh with Scrapy, therefore I have the following questions:
- Can I use proxymesh with ScrapingHub (Proxymesh requires a fixed IP address from the requests). Does ScrapingHub provide a unique IP address for a project or does it change dynamically every request / day / month?
- If not, how many IP addresses does Crawlera provide?
- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.
- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.
Hi,
I am migrating my Scrapy project to ScrapingHub.
I am currently using Proxymesh with Scrapy, therefore I have the following questions:
- Can I use proxymesh with ScrapingHub (Proxymesh requires a fixed IP address from the requests). Does ScrapingHub provide a unique IP address for a project or does it change dynamically every request / day / month?
- If not, how many IP addresses does Crawlera provide?
0 Votes
nestor posted almost 7 years ago Admin Best Answer
- Scrapy Cloud IPs are dynamic and can change, we can't provide a specific range. Please see: https://support.scrapinghub.com/support/solutions/articles/22000200422-what-are-the-ip-addresses-used-by-scrapy-cloud-. I would suggest an authentication (username:password) approach to authenticate to Proxymesh, if possible.
- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.
0 Votes
1 Comments
nestor posted almost 7 years ago Admin Answer
- Scrapy Cloud IPs are dynamic and can change, we can't provide a specific range. Please see: https://support.scrapinghub.com/support/solutions/articles/22000200422-what-are-the-ip-addresses-used-by-scrapy-cloud-. I would suggest an authentication (username:password) approach to authenticate to Proxymesh, if possible.
- Crawlera's number of IP also varies. Our plans start at $25/month for 150,000 monthly requests (https://scrapinghub.com/crawlera), these plans are based on Monthly requests limits and maximum concurrent requests, so the number of IPs should not matter.
0 Votes
Login to post a comment