Learn all about the latest trends and best practices in data extraction - Join us at Extract SummitGet tickets
Start a new topic

error running storage.scrapinghub.com/items

Hi!


I't tryng put in code the curl command below:

curl -u <APIKEY> "https://storage.scrapinghub.com/items/010140/1/24870"


on golang

req,err := http.NewRequest("GET", "https://storage.scrapinghub.com/items/"+jobId, nil)

req.setBasicAuth(key,"")

req.Do()


I got the Unauthorized message


I'm using golang to send a job request and consult the job status (both work).. but to get itens fromĀ the job finished isn't work.


I changed GET to POST, but no success



This "https://storage.scrapinghub.com/items/010140/1/24870" doesn't seem to be a valid job id.


What are you putting as jobId in your GoLang script? It seems like you're getting a redirect and GoLang doesn't propagate BasicAuth to the followed redirect.


1 person likes this

I'm putting the id of the job ( xxx/x/xxxx). The var jobId works when I call:

exec.Command("curl", "-u",key, "https://storage.scrapinghub.com/items/"+jobId)


Try setting Basic Auth as a header instead, with base64 encoded "apikey:" as opposed to SetBasicAuth. Without more details about what how you're capturing the value of jobID and a real sample ID, I can't provide more information.
I can only infer what I mentioned in my last message: "It seems like you're getting a redirect and GoLang doesn't propagate SetBasicAuth to the followed redirect."

Login to post a comment