Handling authentication redirects - oauth2

Posted over 7 years ago by smartcell

Post a topic

Answered

smartcell

Hi,

We have a script using robobrowser that can follow a login form without an issue, but when we add Crawlera (using a Crawlera Session as well), the redirects aren't followed properly and we can't login. The website requires OAuth2 authentication, which might be what is tripping up Crawlera.

Has anyone encountered a similar situation or know how to handle it?

Thanks

0 Votes

smartcell posted over 7 years ago Best Answer

In case someone runs into the same issue: while using the python library requests, you need to pass the API key into the URL for the proxy rather than use HTTPProxyAuth. Otherwise, the api key isn't properly passed by requests when HTTPS redirects happen.

This is the way to go:

proxy_auth = "<APIKEY>:" proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port), "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}

2 Votes

2 Comments

smartcell posted over 7 years ago Answer

This is the way to go:

proxy_auth = "<APIKEY>:" proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port), "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}

2 Votes

Pablo Hoffman posted over 7 years ago Admin

Thanks for sharing!

0 Votes