Hello.
as explained here, I have a spider that takes a very long time to process an item (it's a PDF being OCR'd by tesseract).
After one hour of OCR, the spider received a SIGTERM -- which seems consistent with this answer.
Can I avoid this using some kind of signal? Perhaps spider_idle?
Thanks
mredaelli
Hello.
as explained here, I have a spider that takes a very long time to process an item (it's a PDF being OCR'd by tesseract).
After one hour of OCR, the spider received a SIGTERM -- which seems consistent with this answer.
Can I avoid this using some kind of signal? Perhaps spider_idle?
Thanks