I want to use Selenium on Scrapy Cloud and I believe my only obstacle at this point is knowing where it is located or installing it if it's not there. I get this error
I feel like it should be enough to use something like this
self.borwser=webdriver.Firefox()
but I get this error
Could not find firefox in your system PATH. Please specify the firefox binary location or install firefox
0 Votes
vaz posted
over 7 years ago
Best Answer
Hi, to use Selenium you need to deploy your own custom docker image. Please check the following documentation:
Hello, it says this topic is over a year old but I was wondering if you managed to solve your problem? I also get that same exact error and finding a way to run firefox on scrapy cloud would really help me. I have been stuck with this for days.
0 Votes
J
Jim Barnesposted
almost 7 years ago
Great - Just sent you an Email.. BTW my name is Jim
0 Votes
h
hoozits728posted
almost 7 years ago
Please contact me at [email protected]. I'd like to try to help. My name is Matt.
0 Votes
J
Jim Barnesposted
almost 7 years ago
If your open to a task we would be happy to employ you to build the image and test on a paid account.
Contact me if you are interested
BTW: We use everything today and were one of the earliest SH clients. We moved away because of the size and scope of the data operations we have. It was not until Dec 2017 we decided to re-evaluate and see if the major pieces were in place we needed.
0 Votes
h
hoozits728posted
almost 7 years ago
Have you tried deploying your own custom docker image? I believe that is the way to go. There is help online I followed but got stuck once I realized I need to pay for it. Otherwise, maybe try beautiful soup.
0 Votes
J
Jim Barnesposted
almost 7 years ago
We are a paying customer and would love to PAY MORE but have to fall back to ScrapyD and other options.
0 Votes
h
hoozits728posted
almost 7 years ago
I worked my tail off to get it to work, too, only to find out that I need to be a paying member for the needed steps to have any effect.
0 Votes
J
Jim Barnesposted
almost 7 years ago
Hi,
We have been working over 1 week solving endless bugs and hurdles to get Selenium with Crawlera and SH working.
All links explored Posts in Stack Exchange and Overflow.
Has anyone actually solved this in a LIVE working scenario?
If SH wanted to solve this you could easily provide a generic image via GitHub which would help the business expand and save a lot of effort.
0 Votes
h
hoozits728posted
over 7 years ago
Thank you. The documentation says it is only available to paying customers. Is this true?
0 Votes
vazposted
over 7 years ago
Answer
Hi, to use Selenium you need to deploy your own custom docker image. Please check the following documentation:
I want to use Selenium on Scrapy Cloud and I believe my only obstacle at this point is knowing where it is located or installing it if it's not there. I get this error
I feel like it should be enough to use something like this
self.borwser=webdriver.Firefox()
but I get this error
Could not find firefox in your system PATH. Please specify the firefox binary location or install firefox
0 Votes
vaz posted over 7 years ago Best Answer
Hi, to use Selenium you need to deploy your own custom docker image. Please check the following documentation:
https://shub.readthedocs.io/en/stable/deploy-custom-image.html#deploy-custom-image
Best regards!
Pablo
0 Votes
10 Comments
harlvl posted almost 6 years ago
Hello, it says this topic is over a year old but I was wondering if you managed to solve your problem? I also get that same exact error and finding a way to run firefox on scrapy cloud would really help me. I have been stuck with this for days.
0 Votes
Jim Barnes posted almost 7 years ago
Great - Just sent you an Email.. BTW my name is Jim
0 Votes
hoozits728 posted almost 7 years ago
Please contact me at [email protected]. I'd like to try to help. My name is Matt.
0 Votes
Jim Barnes posted almost 7 years ago
If your open to a task we would be happy to employ you to build the image and test on a paid account.
Contact me if you are interested
BTW: We use everything today and were one of the earliest SH clients. We moved away because of the size and scope of the data operations we have. It was not until Dec 2017 we decided to re-evaluate and see if the major pieces were in place we needed.
0 Votes
hoozits728 posted almost 7 years ago
Have you tried deploying your own custom docker image? I believe that is the way to go. There is help online I followed but got stuck once I realized I need to pay for it. Otherwise, maybe try beautiful soup.
0 Votes
Jim Barnes posted almost 7 years ago
We are a paying customer and would love to PAY MORE but have to fall back to ScrapyD and other options.
0 Votes
hoozits728 posted almost 7 years ago
I worked my tail off to get it to work, too, only to find out that I need to be a paying member for the needed steps to have any effect.
0 Votes
Jim Barnes posted almost 7 years ago
Hi,
We have been working over 1 week solving endless bugs and hurdles to get Selenium with Crawlera and SH working.
All links explored Posts in Stack Exchange and Overflow.
Has anyone actually solved this in a LIVE working scenario?
If SH wanted to solve this you could easily provide a generic image via GitHub which would help the business expand and save a lot of effort.
0 Votes
hoozits728 posted over 7 years ago
Thank you. The documentation says it is only available to paying customers. Is this true?
0 Votes
vaz posted over 7 years ago Answer
Hi, to use Selenium you need to deploy your own custom docker image. Please check the following documentation:
https://shub.readthedocs.io/en/stable/deploy-custom-image.html#deploy-custom-image
Best regards!
Pablo
0 Votes
Login to post a comment