Zyte Support Center

I use the content from the storage URL (https://storage.scrapinghub.com/activity/nnnnnn/n/nnn?count=1&apikey=[api key]) to know when my task has finished. After starting the crawl, I poll the url every few seconds to check it.

I have just noticed that this isn't being updated in all cases when the job finishes.

It should say e.g.

{"job":"3****4/1/315","event":"job:completed","user":"jobrunner"}

when completed, but it is still reporting e.g.

{"job":"3****1/3/392","event":"job:started","user":"jobrunner"}

tens of minutes after the job has completed.

This all worked fine until 13th Sept, and now doesn't. Any ideas, please?

I have done some more research and re-posted this under a better title.