Error 404 when using Session ID

Posted about 6 years ago by erlans

Post a topic

Un Answered

erlans

Hi,

I got 404 error for my second request when i use session id.

I am creating Session Id with my first request

headers={'X-Crawlera-Session': 'create'},

getting session id in response

session_id = response.headers.get('X-Crawlera-Session', '')

and sending second request

scrapy.FormRequest.from_response(

response,

headers={'X-Crawlera-Session': session_id},

formdata={...},

callback=self.parseResult)

When crawlera is disabled, everything works fine

0 Votes

3 Comments

nestor posted about 6 years ago Admin

I see it's POST request. Try adding the header "X-Crawlera-Cookies: disable". If that doesn't work, I would suggest you open a ticket (Help > Contact Support) and provide the data you are sending on the request.

0 Votes

erlans posted about 6 years ago

I got --> [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404...>: HTTP status code is not handled or not allowed

Everything works fine when crawlera is disabled

0 Votes

nestor posted about 6 years ago Admin

A 404 typically means URL not found, are you sure that's the error code you're getting?

0 Votes