I've been trying to run an spider on Scrapy Cloud and this error "Unhandled error in Deferred" is raised, I don't know why. The spider is working well in local.
Jerick Órdenes Sepúlveda posted
over 7 years ago
Best Answer
Yes, the scraper finally worked. I was propagating a response (I needed to do it) using the meta key, it worked in local. I solved the problem passing the response using functools.partial.
0 Votes
3 Comments
Sorted by
T
Takeaki Kawaiposted
almost 6 years ago
I got the same issue.
When I passed the ItemLoader object to callback function,
with passing the response object, it works on local but not on Scrapinghub.
Jerick Órdenes Sepúlvedaposted
over 7 years ago
Answer
Yes, the scraper finally worked. I was propagating a response (I needed to do it) using the meta key, it worked in local. I solved the problem passing the response using functools.partial.
0 Votes
vazposted
over 7 years ago
Did the spider extracted any elements beyond the error?
Try to check if all your file resources in locally are the same as in Scrapy cloud.
I've been trying to run an spider on Scrapy Cloud and this error "Unhandled error in Deferred" is raised, I don't know why. The spider is working well in local.
Attachments (1)
logtommyhilf....txt
158 KB
0 Votes
Jerick Órdenes Sepúlveda posted over 7 years ago Best Answer
Yes, the scraper finally worked. I was propagating a response (I needed to do it) using the meta key, it worked in local. I solved the problem passing the response using functools.partial.
0 Votes
3 Comments
Takeaki Kawai posted almost 6 years ago
I got the same issue.
When I passed the ItemLoader object to callback function,
with passing the response object, it works on local but not on Scrapinghub.
Make sure use
not like
when you deploy.
Scrapy: Requests and Responses
0 Votes
Jerick Órdenes Sepúlveda posted over 7 years ago Answer
Yes, the scraper finally worked. I was propagating a response (I needed to do it) using the meta key, it worked in local. I solved the problem passing the response using functools.partial.
0 Votes
vaz posted over 7 years ago
Did the spider extracted any elements beyond the error?
Try to check if all your file resources in locally are the same as in Scrapy cloud.
Best,
Pablo
0 Votes
Login to post a comment