Start a new topic

Is it possible to resume jobs?

On the official Scrapy docs there is a section for "pausing and resuming crawls". It states that through the JOBDIR setting it is possible to resume the state of a job. I've also found a  Zyte's FAQ that says "JOBDIR is set, causing the scheduler (requests queue) to be persisted on disk, and save memory".

Is there a way to override JOBDIR in order to resume a stopped job from where it terminated its execution?

Login to post a comment