How can we help you today?

Start a new topic
Discussions Zyte Smart Proxy Manager (formerly Crawlera) Technical Help
Answered

Spider arguments ignored

Hi, I am having a similar issue as the one described here: https://support.scrapinghub.com/support/discussions/topics/22000009043

and I have tried the suggestions listed there but they don't work.


Running locally, the command line arguments are passed to the from_crawler method as expected but not when running from Scraping Hub.


What can I do to resolve this?

Best Answer

Hello,



You can pass Arguments while running the Jon through UI (Arguments section of the Run pop-up) as shown below or cURL data option if running a job via API.


3 CommentsSorted by Oldest First
Answer

Hello,



You can pass Arguments while running the Jon through UI (Arguments section of the Run pop-up) as shown below or cURL data option if running a job via API.


1 person likes this

Thank you very much! The arguments are passed correctly now.

However, I'm still confused as to how to access the settings defined on the spider settings page.

Glad to know that Arguments worked :)


For accessing the Spider settings, you would need to navigate to setting page of the spider like example https://app.scrapinghub.com/p/444534/3/settings and then use the (+),(-) and add or modify the settings. Please refer https://support.scrapinghub.com/a/solutions/articles/22000200670 to know about spider settings page. 

Login to post a comment
More topics in Technical Help
See all 389 topics