Start a new topic

HTTP Request doesn't work with WebProxy

 

I'm scraping data from some sites and one of theses use headers for some information. I'm using Smart Proxy Manager from Zyte and having problems with a request with headers. When I use the zyte proxy, I receive a response, but the site that I'm scraping return an unsuccess message. If I not use the proxy, the request works like a charm. Can anyone help me?

Here is my code

string zyteKey = _configuration["ZyteKey"];
            var proxy = new WebProxy("myproxy");
            proxy.Credentials = new NetworkCredential(zyteKey, "");

            var clientHandler = new HttpClientHandler()
            {
                Proxy = proxy,
                UseProxy = true,
                PreAuthenticate = true,
                AllowAutoRedirect = false
            };

            using (var client = new HttpClient(clientHandler))
            {
                string json = JsonConvert.SerializeObject(jsonObject);
                var content = new StringContent(json, Encoding.UTF8, "application/json");

                client.Timeout = new TimeSpan(0, 1, 0);
                client.DefaultRequestHeaders.Add("cookie", BuildCookieRequest(latitude, longitude, restaurantName));
                client.DefaultRequestHeaders.Add("Accept", "application/json, text/plain, */*");
                client.DefaultRequestHeaders.Add("x-csrf-token", "x");

                return await client.PostAsync(url, content);
            }

Login to post a comment