Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic
Answered

ImportError: No module named scrapy_user_agents.middlewares

Hi,

when I run my spider locally it runs without any problems. But on the scrapycloud I got the following error Message:

ImportError: No module named scrapy_user_agents.middlewares

Any idea what I have to change in my settings or code? 


thanks.


Best Answer

Answer

Hi, You would need to add the library to your requirements.txt as shown here:  https://support.scrapinghub.com/support/solutions/articles/22000200400-deploying-python-dependencies-for-your-projects-in-scrapy-cloud 

I have the exact same issue, how did you end up solving this? I'm not sure what to add to my requirements.txt :(

Login to post a comment