I am new to the Scrapinghub, do you guys know if there is a plug-in for storing the crawling data on the fly into hosted Elasticsearch @ elastic cloud? FYI. I did this well with Scrapy item pipeline setting @ my desktop. Thanks.
0 Votes
Nestor Toledo posted
over 7 years ago
AdminBest Answer
I am new to the Scrapinghub, do you guys know if there is a plug-in for storing the crawling data on the fly into hosted Elasticsearch @ elastic cloud? FYI. I did this well with Scrapy item pipeline setting @ my desktop. Thanks.
0 Votes
Nestor Toledo posted over 7 years ago Admin Best Answer
The same pipeline should work in Scrapy Cloud as long as you provide all the required python dependencies in your requirements.txt. See: https://support.scrapinghub.com/support/solutions/articles/22000200400-deploying-python-dependencies-for-your-projects-in-scrapy-cloud
0 Votes
1 Comments
Nestor Toledo posted over 7 years ago Admin Answer
The same pipeline should work in Scrapy Cloud as long as you provide all the required python dependencies in your requirements.txt. See: https://support.scrapinghub.com/support/solutions/articles/22000200400-deploying-python-dependencies-for-your-projects-in-scrapy-cloud
0 Votes
Login to post a comment