videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

How do I call list jobs in a paginated fashion?

I tried: 

https://app.scrapinghub.com/api/jobs/list.json?project=<project-num?&spider=<spider-name>&state=finished&start=125

  as well as:

https://app.scrapinghub.com/api/jobs/list.json?project=<project-num?&spider=<spider-name>&state=finished&start=<project-id>/1/125

But neither returned anything but the most recent 100 results. How do I get older results?


Best Answer

Hi,


for list.json the pagination keyword is "offset" instead of "start".


Answer

Hi,


for list.json the pagination keyword is "offset" instead of "start".


1 person likes this

Thanks! Worked like a charm. Can this be documented?

Login to post a comment