I'm scraping data from some sites and one of theses use headers for some information. I'm using Smart Proxy Manager from Zyte and having problems with a request with headers. When I use the zyte proxy, I receive a response, but the site that I'm scraping return an unsuccess message. If I not use the proxy, the request works like a charm. Can anyone help me?
Here is my code
string zyteKey = _configuration["ZyteKey"];
var proxy = new WebProxy("myproxy");
proxy.Credentials = new NetworkCredential(zyteKey, "");
var clientHandler = new HttpClientHandler()
{
Proxy = proxy,
UseProxy = true,
PreAuthenticate = true,
AllowAutoRedirect = false
};
using (var client = new HttpClient(clientHandler))
{
string json = JsonConvert.SerializeObject(jsonObject);
var content = new StringContent(json, Encoding.UTF8, "application/json");
client.Timeout = new TimeSpan(0, 1, 0);
client.DefaultRequestHeaders.Add("cookie", BuildCookieRequest(latitude, longitude, restaurantName));
client.DefaultRequestHeaders.Add("Accept", "application/json, text/plain, */*");
client.DefaultRequestHeaders.Add("x-csrf-token", "x");
returnawait client.PostAsync(url, content);
}
I'm scraping data from some sites and one of theses use headers for some information. I'm using Smart Proxy Manager from Zyte and having problems with a request with headers. When I use the zyte proxy, I receive a response, but the site that I'm scraping return an unsuccess message. If I not use the proxy, the request works like a charm. Can anyone help me?
Here is my code
0 Votes
0 Comments
Login to post a comment