I want to run a job using jobs API.
I am passing the project id and the spider name.
I would like to pass arguments to the spider as well.
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
$ curl -u APIKEY: https://app.scrapinghub.com/api/run.json -d project=123 -d spider=somespider -d spiderarg1=example
me too
how to capture the argument spiderarg1=example in this spider?
Can someone tell me how I can access those argument passed to the Job?
Why i continuesly get "invalid value for project" error.
I have passed correct arguments and API
curl -u API: "https://app.scrapinghub.com/api/jobs/list.json?project=tutorial'&spider='quotes'"
The arguments can be captured in the __init__ method and can be used as attributes. Please refer https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments to know how to use the arguments.
hi @bgt-karthi,
The project number should be provided in the URL as
$ curl -u APIKEY: "https://app.scrapinghub.com/api/jobs/list.json?project=123&spider=somespider". Please provide the spidername without the single quotes.
Avishay Balderman
I want to run a job using jobs API.
I am passing the project id and the spider name.
I would like to pass arguments to the spider as well.
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
- Oldest First
- Popular
- Newest First
Sorted by Oldest FirstMarcos Machado
me too
nestor
Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:
1 person likes this
saptadiparc
how to capture the argument spiderarg1=example in this spider?
Navneet Verma
Can someone tell me how I can access those argument passed to the Job?
bgt-karthi
Why i continuesly get "invalid value for project" error.
I have passed correct arguments and API
curl -u API: "https://app.scrapinghub.com/api/jobs/list.json?project=tutorial'&spider='quotes'"
thriveni
The arguments can be captured in the __init__ method and can be used as attributes. Please refer https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments to know how to use the arguments.
thriveni
hi @bgt-karthi,
The project number should be provided in the URL as
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 438 topics