Hi.
i have 2 spiders - Spider1 and Spider2. i want to run Spider2 after completion of Spider1. The scheduling doesn't help me much because i won't to start Spider2 immediately after Spider1.
Is there any easy way to do so?
You can use Spider signals to schedule a new job https://docs.scrapy.org/en/latest/topics/signals.html#spider-closed, on spider_closed for spider1 you can use the Jobs API to schedule spider2.
Sarkis Matinyan
Hi.
i have 2 spiders - Spider1 and Spider2. i want to run Spider2 after completion of Spider1. The scheduling doesn't help me much because i won't to start Spider2 immediately after Spider1.
Is there any easy way to do so?
You can use Spider signals to schedule a new job https://docs.scrapy.org/en/latest/topics/signals.html#spider-closed, on spider_closed for spider1 you can use the Jobs API to schedule spider2.
nestor
You can use Spider signals to schedule a new job https://docs.scrapy.org/en/latest/topics/signals.html#spider-closed, on spider_closed for spider1 you can use the Jobs API to schedule spider2.
1 person likes this
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 442 topics