3 things to evaluate to discover the best way for you to access web data.Register for the webinar
Start a new topic

Github Deploy

I require to run 100+ spiders of different websites from Github, I would like to know what would be the best way to deploy them,

Multiple Repos, Multiple branch ?


If single master branch how should the 100+ spiders should be saved, different projects, single project different name?


Would it require 100s of scrapy cfg?


Thank you! 



Login to post a comment