def parse(self, response): # for testing single practice link # return self.parse_practice(response) for practice_row in response.css('table.listing-display tr'): practice_links = practice_row.css('a::attr(href)') if len(practice_links) > 0: yield response.follow(practice_links[0], self.parse_practice) ```
Why is it breaking on `response.follow(...)` ? it works fine locally just not on scrapinghub.com.
Local running Scrapy version 1.4.0
Best Answer
t
thriveni
said
almost 7 years ago
The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks
The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks
jward
```
def parse(self, response):
# for testing single practice link
# return self.parse_practice(response)
for practice_row in response.css('table.listing-display tr'):
practice_links = practice_row.css('a::attr(href)')
if len(practice_links) > 0:
yield response.follow(practice_links[0], self.parse_practice)
```
Why is it breaking on `response.follow(...)` ? it works fine locally just not on scrapinghub.com.
Local running Scrapy version 1.4.0
The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks
- Oldest First
- Popular
- Newest First
Sorted by Popularthriveni
The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks
1 person likes this
jward
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 460 topics