Saving a a scrapingcloud deployed crawl to Database

Posted about 7 years ago by dubarseo

Post a topic
Un Answered
d
dubarseo

Hi there :) 


With the adaptated pipeline, is this possible to save our crawl result to an SQL database ? I wonder if scrapy-cloud use his own pipeline or the one we made in pipeline.py


Regards 

0 Votes


1 Comments

tom

tom posted about 7 years ago Admin

Hi yea you can do something like https://docs.scrapy.org/en/latest/topics/item-pipeline.html#write-items-to-mongodb. Scrapy cloud would just use what you have set in your spider.

0 Votes

Login to post a comment