How can we help you today?

Start a new topic
Discussions Scrapy Cloud Technical Help
Answered

Store crawled data into hosted Elasticsearch @ elastic cloud

I am new to the Scrapinghub, do you guys know if there is a plug-in for storing the crawling data on the fly into hosted Elasticsearch @ elastic cloud? FYI. I did this well with Scrapy item pipeline setting @ my desktop. Thanks.

Best Answer

The same pipeline should work in Scrapy Cloud as long as you provide all the required python dependencies in your requirements.txt. See: https://support.scrapinghub.com/support/solutions/articles/22000200400-deploying-python-dependencies-for-your-projects-in-scrapy-cloud

1 Comment
Answer

The same pipeline should work in Scrapy Cloud as long as you provide all the required python dependencies in your requirements.txt. See: https://support.scrapinghub.com/support/solutions/articles/22000200400-deploying-python-dependencies-for-your-projects-in-scrapy-cloud

Login to post a comment
More topics in Technical Help
See all 438 topics