Start a new topic

Spider terminated after one hour


as explained here, I have a spider that takes a very long time to process an item (it's a PDF being OCR'd by tesseract).

After one hour of OCR, the spider received a SIGTERM -- which seems consistent with this answer.

Can I avoid this using some kind of signal? Perhaps spider_idle?


Login to post a comment