Why do I still see jobs older than a day after setting the project's data retention period to one day?

Modified on Wed, 3 Feb, 2021 at 8:33 AM

The job data remains available to ensure it can be restored if needed. The jobs will be marked as deleted, but not removed immediately for this very reason (like moving a file to trash) -- the actual data is wiped 7 days later.


Jobs are deleted daily, but the batch size is limited, so it's normal to see jobs up to a day older than specified in the data retention settings.


Please also refer to this answer on StackOverflow: https://stackoverflow.com/questions/56529737/what-happens-after-scrapy-job-is-deleted-before-it-is-erased/56539708#56539708.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article