Zyte Support Center

Hi,

We have a script using robobrowser that can follow a login form without an issue, but when we add Crawlera (using a Crawlera Session as well), the redirects aren't followed properly and we can't login. The website requires OAuth2 authentication, which might be what is tripping up Crawlera.

Has anyone encountered a similar situation or know how to handle it?

Thanks

In case someone runs into the same issue: while using the python library requests, you need to pass the API key into the URL for the proxy rather than use HTTPProxyAuth. Otherwise, the api key isn't properly passed by requests when HTTPS redirects happen.

This is the way to go:

proxy_auth = "<APIKEY>:" proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port), "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}