In some occasions you may experience errors using DeltaFetch due the interactions with files in S3.
Your output may show errors like this:
DBRunRecoveryError: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB0060 PANIC: fatal region error detected; run recovery')"}]
The easiest way to correct this error is to reset the .db file.
The procedure to reset can be found here: https://github.com/scrapy-plugins/scrapy-deltafetch#supported-scrapy-settings
Please follow these steps:
- Set in your spider: (spider settings > `DELTAFETCH_RESET: 1`)
- Run spider again
- Disable setting
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article