Can I run the same spider in parallel?

Modified on Wed, 3 Feb, 2021 at 8:31 AM

You can, provided that you have more than one Scrapy Cloud unit and the spider's arguments differ (entered, for instance, through Arguments section of the Run pop-up in Scrapy Cloud UI, or cURL data option if running a job via API).

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article