In some occasions you may experience errors using DeltaFetch due the interactions with files in S3.
Your output may show errors like this:
DBRunRecoveryError: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB0060 PANIC: fatal region error detected; run recovery')"}]
The easiest way to correct this error is to reset the .db file.
The procedure to reset can be found here: https://github.com/scrapy-plugins/scrapy-deltafetch#supported-scrapy-settings
Please follow these steps:
- Set in your spider: (spider settings > `DELTAFETCH_RESET: 1`)
- Run spider again
- Disable setting