Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic
Answered

HtmlResponse' object has no attribute 'follow'

 ```

    def parse(self, response):
        # for testing single practice link
        # return self.parse_practice(response)
        for practice_row in response.css('table.listing-display tr'):
            practice_links = practice_row.css('a::attr(href)')
            if len(practice_links) > 0:
               yield response.follow(practice_links[0], self.parse_practice)
```


Why is it breaking on `response.follow(...)` ? it works fine locally just not on scrapinghub.com.

Local running Scrapy version 1.4.0


Best Answer

The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks  


Answer

The error seems to be due to different Scrapy versions. The deploy to Scrapinghub has been done using Scrapy stack 1.1. You would need to change the stack in your projects scrapinghub .yml as given in article Changing the deploy environment with Scrapy Cloud Stacks  


1 person likes this
Thanks thriveni!

 

Login to post a comment