How can we help you today?

Start a new topic
Discussions Scrapy Cloud Technical Help
Answered

Frontera: Store URLS between pauses

Hello,

Does Frontera allow to store the scheduled URLs? For example, using scrapy_redis, if the crawler has run before, scheduled URLs are stored when the crawler is paused/canceled. This allows that the crawler can continue where it was left. If Frontera supports this, how can I implement it?


Thank you

Best Answer

Please post Frontera related questions in Frontera's Google group: https://groups.google.com/a/scrapinghub.com/forum/#!forum/frontera or check it's main documentation: http://frontera.readthedocs.org/ and/or Github Issues/Pull requests: https://github.com/scrapinghub/frontera.

1 Comment
Answer

Please post Frontera related questions in Frontera's Google group: https://groups.google.com/a/scrapinghub.com/forum/#!forum/frontera or check it's main documentation: http://frontera.readthedocs.org/ and/or Github Issues/Pull requests: https://github.com/scrapinghub/frontera.

Comments to this discussion are now closed!
More topics in Technical Help
See all 453 topics