200 response but no results

Posted over 5 years ago by recruitlocator040116

Post a topic
Un Answered
r
recruitlocator040116

I've been experiencing issues off and on with receiving results. Even in instances where the crawlera console shows 200 response and valid results, they are not rendering in the client.  Out of 10 instances where the query returns 200 responses, I might get results to render in 1 or 2 cases.  I've tried making changes to the CURLOPT_TIMEOUT, X-Crawlera-Timeout, and X-Crawlera-Max-Retries with no improvements.  

		$ch = curl_init();
		curl_setopt($ch, CURLOPT_URL, $url);
		//curl_setopt($ch, CURLOPT_POST, 1);
		//curl_setopt($ch, CURLOPT_POSTFIELDS, $query);
		curl_setopt($ch, CURLOPT_CAINFO, $this->cert);
		curl_setopt($ch, CURLOPT_PROXY, $this->proxy);
		curl_setopt($ch, CURLOPT_PROXYUSERPWD, $this->userpwd);
		//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
		//curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
		curl_setopt($ch, CURLOPT_TIMEOUT, 80);
        curl_setopt($ch, CURLOPT_HTTPHEADER, array(
			    'X-Crawlera-Profile:mobile',
                'X-Crawlera-Cookies:disable',
                'X-Crawlera-Max-Retries:1',
				'X-Crawlera-Timeout:60000',
				'Accept-Encoding:identity,gzip,deflate',
                ));
		curl_setopt($ch, CURLOPT_VERBOSE, true);
		curl_setopt($ch, CURLOPT_STDERR, $fp);

		$response = curl_exec ($ch);
		curl_close ($ch);
		//fclose($fp);

   

0 Votes


0 Comments

Login to post a comment