How to use Hooks in Python Requests (when using Crawlera)
Posted almost 7 years ago by Graham Anderson
Post a topicPeople who like this
Delete Comment
This post will be deleted permanently. Are you sure?
Code Snippet
Html
Html
Css
JavaScript
Sass
Xml
Ruby
PHP
Java
C#
C++
ObjectiveC
Perl
Python
VB
SQL
Generic Language
Is this the proper way to use hooks with crawlera...so requests keeps trying until it the request is successful?
I'm no scraping expert. Any mods/improvements/deletions is welcome as I'm trying to get a little better at this....
Questions/Confusion:
Re the response, what should url_check variable be? Am I supposed to pass in a known/working url from the crawl site...or am I supposed to pass the actual url being crawled?
if its the actual url, how do you pass that variable into the check_status function?
Usage:
0 Votes
0 Comments
Login to post a comment