Question: Is API always returning response without 'items' key when there are 0 jobs? No matter if script failed because of an error/ 500 responses or just finished and no items were found?
The API returns only the fields that have been updated during runtime, it doesn't have a predefined set of fields, this is why items key is not returned if the item count is 0.
I've created a ticket as a feature request, but for now, if you need that key with 0 value then you will need to add it manually.
0 Votes
2 Comments
Sorted by
L
Lukasz Jaworowskiposted
about 6 years ago
Thank you, It's not urgent. I just wanted to make sure that it does not happen only in specific case like job finished because of error or something like this.
0 Votes
nestorposted
about 6 years ago
AdminAnswer
The API returns only the fields that have been updated during runtime, it doesn't have a predefined set of fields, this is why items key is not returned if the item count is 0.
I've created a ticket as a feature request, but for now, if you need that key with 0 value then you will need to add it manually.
Hi,
I am using storage.scrapinghub.com/jobq/{proj_id}/lis to get list of jobs finished in specific period of time.
I am using startts, endts and state=finished filter.
The problem is sometimes 'items' key is missing.
Examples:
1 When there are items scraped.
{'close_reason': 'finished', 'pending_time': XXXX, 'spider': 'XXX', 'running_time': XXXX, 'state': 'finished', 'version': 'XXXX', 'finished_time': XXXX, 'ts': XXXX, 'elapsed': XXXX, 'key': 'XXXX/XXXX/XXXX', 'items': 18, 'pages': 1, 'logs': 17}
2 When there are no items scraped?
{'version': 'XXX', 'finished_time': XX, 'state': 'XXX', 'pending_time': XX, 'running_time': XXX, 'close_reason': 'finished', 'spider': 'XXX', 'ts': XXXX, 'elapsed': XXX, 'key': 'XXX/XX/XX', 'logs': 17, 'pages': 2} (no 'items' key)
Question: Is API always returning response without 'items' key when there are 0 jobs? No matter if script failed because of an error/ 500 responses or just finished and no items were found?
Is it impossible to get 'items':0 response?
There is no info about that field in official api https://doc.scrapinghub.com/api/jobq.html#jobq-project-id-list
0 Votes
nestor posted about 6 years ago Admin Best Answer
The API returns only the fields that have been updated during runtime, it doesn't have a predefined set of fields, this is why items key is not returned if the item count is 0.
I've created a ticket as a feature request, but for now, if you need that key with 0 value then you will need to add it manually.
0 Votes
2 Comments
Lukasz Jaworowski posted about 6 years ago
Thank you, It's not urgent. I just wanted to make sure that it does not happen only in specific case like job finished because of error or something like this.
0 Votes
nestor posted about 6 years ago Admin Answer
The API returns only the fields that have been updated during runtime, it doesn't have a predefined set of fields, this is why items key is not returned if the item count is 0.
I've created a ticket as a feature request, but for now, if you need that key with 0 value then you will need to add it manually.
0 Votes
Login to post a comment