Start a new topic

Saving a a scrapingcloud deployed crawl to Database

Hi there :) 


With the adaptated pipeline, is this possible to save our crawl result to an SQL database ? I wonder if scrapy-cloud use his own pipeline or the one we made in pipeline.py


Regards 

1 Comment

Hi yea you can do something like https://docs.scrapy.org/en/latest/topics/item-pipeline.html#write-items-to-mongodb. Scrapy cloud would just use what you have set in your spider.

Login to post a comment