I'm trying to scrap urls from a site that is hosted on cloudflare using python requests module and then using selenium.
In both cases, I used proxy urls to scrap data. But somehow they are recognizing the requests as bots and responded with Captcha. Seems they are differentiating bots and humans using the cookies passed in the request header.
I'm planning to use Zyte proxies. My question is whether by using Zyte proxies, I'll be able to bypass the issues of Captcha and bot recognition.
Ajith Vijayan
Hi,
I'm trying to scrap urls from a site that is hosted on cloudflare using python requests module and then using selenium.
In both cases, I used proxy urls to scrap data. But somehow they are recognizing the requests as bots and responded with Captcha. Seems they are differentiating bots and humans using the cookies passed in the request header.
I'm planning to use Zyte proxies. My question is whether by using Zyte proxies, I'll be able to bypass the issues of Captcha and bot recognition.
My client is desperately in need of a solution.