videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

How to setup Crawlera in Firefox Selenium python?

def firefox_crawlera():

 profile = webdriver.FirefoxProfile() 

    profile.set_preference("network.proxy.type", 1)

    profile.set_preference("network.proxy.http", "CRAWLERA_APIKEY@proxy.crawlera.com")

    profile.set_preference("network.proxy.http_port", 8010)

    profile.update_preferences()

    driver = webdriver.Firefox(firefox_profile=profile, executable_path=r'c:\dropbox\py\webdriver\geckodriver.exe')

    return driver


Proxy doesn't work.



Best Answer

Try using Crawlera-headless-proxy as an alternative to Polipo:

https://github.com/scrapinghub/crawlera-headless-proxy 

1 Comment

Answer

Try using Crawlera-headless-proxy as an alternative to Polipo:

https://github.com/scrapinghub/crawlera-headless-proxy 

Login to post a comment