Does Frontera allow to store the scheduled URLs? For example, using scrapy_redis, if the crawler has run before, scheduled URLs are stored when the crawler is paused/canceled. This allows that the crawler can continue where it was left. If Frontera supports this, how can I implement it?
Hello,
Does Frontera allow to store the scheduled URLs? For example, using scrapy_redis, if the crawler has run before, scheduled URLs are stored when the crawler is paused/canceled. This allows that the crawler can continue where it was left. If Frontera supports this, how can I implement it?
Thank you
0 Votes
nestor posted over 6 years ago Admin Best Answer
Please post Frontera related questions in Frontera's Google group: https://groups.google.com/a/scrapinghub.com/forum/#!forum/frontera or check it's main documentation: http://frontera.readthedocs.org/ and/or Github Issues/Pull requests: https://github.com/scrapinghub/frontera.
0 Votes
1 Comments
nestor posted over 6 years ago Admin Answer
Please post Frontera related questions in Frontera's Google group: https://groups.google.com/a/scrapinghub.com/forum/#!forum/frontera or check it's main documentation: http://frontera.readthedocs.org/ and/or Github Issues/Pull requests: https://github.com/scrapinghub/frontera.
0 Votes