Zyte Support Center
How can we help you today?
Error 404 when using Session ID
3 CommentsSorted by Oldest First
nestor
said
almost 6 years ago
A 404 typically means URL not found, are you sure that's the error code you're getting?
I got --> [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404...>: HTTP status code is not handled or not allowed
Everything works fine when crawlera is disabled
nestor
said
almost 6 years ago
I see it's POST request. Try adding the header "X-Crawlera-Cookies: disable". If that doesn't work, I would suggest you open a ticket (Help > Contact Support) and provide the data you are sending on the request.
erlans
Hi,
I got 404 error for my second request when i use session id.
I am creating Session Id with my first request
headers={'X-Crawlera-Session': 'create'},
getting session id in response
session_id = response.headers.get('X-Crawlera-Session', '')
and sending second request
scrapy.FormRequest.from_response(
response,
headers={'X-Crawlera-Session': session_id},
formdata={...},
callback=self.parseResult)
When crawlera is disabled, everything works fine