No recent searches
Popular Articles
Sorry! nothing found for
Posted over 5 years ago by Mark Salisbury
I use the content from the storage URL (https://storage.scrapinghub.com/activity/nnnnnn/n/nnn?count=1&apikey=[api key]) to know when my task has finished. After starting the crawl, I poll the url every few seconds to check it.
I have just noticed that this isn't being updated in all cases when the job finishes.
It should say e.g.
{"job":"3****4/1/315","event":"job:completed","user":"jobrunner"}
when completed, but it is still reporting e.g.
{"job":"3****1/3/392","event":"job:started","user":"jobrunner"}
tens of minutes after the job has completed.
This all worked fine until 13th Sept, and now doesn't. Any ideas, please?
0 Votes
2 Comments
nestor posted over 5 years ago Admin
Could you provide real examples of jobs that are missing from the Activity API or Activity tab in the UI?
Mark Salisbury posted over 5 years ago
I have done some more research and re-posted this under a better title.
Login to post a comment
People who like this
This post will be deleted permanently. Are you sure?
I use the content from the storage URL (https://storage.scrapinghub.com/activity/nnnnnn/n/nnn?count=1&apikey=[api key]) to know when my task has finished. After starting the crawl, I poll the url every few seconds to check it.
I have just noticed that this isn't being updated in all cases when the job finishes.
It should say e.g.
{"job":"3****4/1/315","event":"job:completed","user":"jobrunner"}
when completed, but it is still reporting e.g.
{"job":"3****1/3/392","event":"job:started","user":"jobrunner"}
tens of minutes after the job has completed.
This all worked fine until 13th Sept, and now doesn't. Any ideas, please?
0 Votes
2 Comments
nestor posted over 5 years ago Admin
Could you provide real examples of jobs that are missing from the Activity API or Activity tab in the UI?
0 Votes
Mark Salisbury posted over 5 years ago
I have done some more research and re-posted this under a better title.
0 Votes
Login to post a comment