I am getting a 451 error when trying to request websites from a specific company. This only occurs when I try to deploy the spider via Scrapinghub, as the spider works just fine when initialized from my computer.
Any ideas on what might cause the difference in performance?
Best Answer
n
nestor
said
over 4 years ago
Our servers are in Germany, that website can only be accessed from the US it seems. You'll need a proxy service with US IPs.
1 Comment
nestor
said
over 4 years ago
Answer
Our servers are in Germany, that website can only be accessed from the US it seems. You'll need a proxy service with US IPs.
valkvent
I am getting a 451 error when trying to request websites from a specific company. This only occurs when I try to deploy the spider via Scrapinghub, as the spider works just fine when initialized from my computer.
Any ideas on what might cause the difference in performance?
Our servers are in Germany, that website can only be accessed from the US it seems. You'll need a proxy service with US IPs.
nestor
Our servers are in Germany, that website can only be accessed from the US it seems. You'll need a proxy service with US IPs.
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 434 topics