Start a new topic
Answered

Handling authentication redirects - oauth2

Hi,


We have a script using robobrowser that can follow a login form without an issue, but when we add Crawlera (using a Crawlera Session as well), the redirects aren't followed properly and we can't login. The website requires OAuth2 authentication, which might be what is tripping up Crawlera.


Has anyone encountered a similar situation or know how to handle it?


Thanks


Best Answer

In case someone runs into the same issue: while using the python library requests, you need to pass the API key into the URL for the proxy rather than use HTTPProxyAuth. Otherwise, the api key isn't properly passed by requests when HTTPS redirects happen.


This is the way to go:

proxy_auth = "<APIKEY>:" proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port), "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}




Answer

In case someone runs into the same issue: while using the python library requests, you need to pass the API key into the URL for the proxy rather than use HTTPProxyAuth. Otherwise, the api key isn't properly passed by requests when HTTPS redirects happen.


This is the way to go:

proxy_auth = "<APIKEY>:" proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port), "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}




2 people like this

Thanks for sharing!

Login to post a comment