No recent searches
Popular Articles
Sorry! nothing found for
Posted over 7 years ago by Avishay Balderman
I want to run a job using jobs API.
I am passing the project id and the spider name.
I would like to pass arguments to the spider as well.
0 Votes
nestor posted about 7 years ago Admin Best Answer
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
$ curl -u APIKEY: https://app.scrapinghub.com/api/run.json -d project=123 -d spider=somespider -d spiderarg1=example
1 Votes
7 Comments
thriveni posted over 4 years ago Admin
hi @bgt-karthi,
The project number should be provided in the URL as
$ curl -u APIKEY: "https://app.scrapinghub.com/api/jobs/list.json?project=123&spider=somespider". Please provide the spidername without the single quotes.
The arguments can be captured in the __init__ method and can be used as attributes. Please refer https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments to know how to use the arguments.
bgt-karthi posted over 4 years ago
Why i continuesly get "invalid value for project" error.
I have passed correct arguments and API
curl -u API: "https://app.scrapinghub.com/api/jobs/list.json?project=tutorial'&spider='quotes'"
Navneet Verma posted over 4 years ago
Can someone tell me how I can access those argument passed to the Job?
saptadiparc posted over 6 years ago
how to capture the argument spiderarg1=example in this spider?
nestor posted about 7 years ago Admin Answer
Marcos Machado posted about 7 years ago
me too
Login to post a comment
People who like this
This post will be deleted permanently. Are you sure?
I want to run a job using jobs API.
I am passing the project id and the spider name.
I would like to pass arguments to the spider as well.
0 Votes
nestor posted about 7 years ago Admin Best Answer
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
1 Votes
7 Comments
thriveni posted over 4 years ago Admin
hi @bgt-karthi,
The project number should be provided in the URL as
0 Votes
thriveni posted over 4 years ago Admin
The arguments can be captured in the __init__ method and can be used as attributes. Please refer https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments to know how to use the arguments.
0 Votes
bgt-karthi posted over 4 years ago
Why i continuesly get "invalid value for project" error.
I have passed correct arguments and API
curl -u API: "https://app.scrapinghub.com/api/jobs/list.json?project=tutorial'&spider='quotes'"
0 Votes
Navneet Verma posted over 4 years ago
Can someone tell me how I can access those argument passed to the Job?
0 Votes
saptadiparc posted over 6 years ago
how to capture the argument spiderarg1=example in this spider?
0 Votes
nestor posted about 7 years ago Admin Answer
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
1 Votes
Marcos Machado posted about 7 years ago
me too
0 Votes
Login to post a comment