I require to run 100+ spiders of different websites from Github, I would like to know what would be the best way to deploy them,
Multiple Repos, Multiple branch ?
If single master branch how should the 100+ spiders should be saved, different projects, single project different name?
Would it require 100s of scrapy cfg?
Thank you!
Darshan Jadav
I require to run 100+ spiders of different websites from Github, I would like to know what would be the best way to deploy them,
Multiple Repos, Multiple branch ?
If single master branch how should the 100+ spiders should be saved, different projects, single project different name?
Would it require 100s of scrapy cfg?
Thank you!