Doing AutoExtract with PHP Curl

Posted over 5 years ago by techytimo

Post a topic

Answered

techytimo

I am trying to extract content from a url with php.

$curl = curl_init();
curl_setopt_array($curl, [
  CURLOPT_HTTPHEADER => [
    'Content-Type: application/json',
    'APIKEY: myapikeymyapikey'
  ],
  CURLOPT_RETURNTRANSFER => 1,
  CURLOPT_URL => 'https://autoextract.scrapinghub.com/v1/extract',
  CURLOPT_POST => 1,
  CURLOPT_POSTFIELDS => [
    'url' => 'https://blog.scrapinghub.com/gopro-study',
    'pageType' => 'article'
  ]
]);
$resp = curl_exec($curl);
curl_close($curl);
return $resp;

I get the error

{

title: "No authentication token provided",

type: "http://errors.xod.scrapinghub.com/unauthorized.html"

}

0 Votes

nestor posted over 5 years ago Admin Best Answer

Try:

    CURLOPT_POSTFIELDS => json_encode([[
        'url' => 'https://blog.scrapinghub.com/gopro-study',
        'pageType' => 'article'
    ]])

0 Votes

15 Comments

promoautotest posted over 4 years ago

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://autoextract.scrapinghub.com/v1/extract');

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($ch, CURLOPT_POST, 1);

curl_setopt($ch, CURLOPT_POSTFIELDS, "[{\"url\": \"https://www.automoto.it/listino\", \"pageType\": \"vehicle\"}]");

curl_setopt($ch, CURLOPT_USERPWD, 'My Key ' . ':' . '');

curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');

$headers = array();

$headers[] = 'Content-Type: application/json';

curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);

$json_string = json_decode($result, true); => couldn't get all car details please help .

0 Votes