videocamWeb Data Extraction Summit - September 30th, 2021.
Join some of the greatest minds in web scraping to educate, inspire, and innovate.
Register for free!
Start a new topic
Answered

Running 'Whois' behind proxy

Linux command 'whois' allows for retrieving data about particular domain age. Unfortunately, there is a limit on the number of operations that can be performed. I was wondering if it is possible to use different IP addresses via Crawlera. Is there any solution?


Best Answer

'whois' runs on TCP protocol, Crawlera is an HTTP proxy so it's not going to work. You can:

A) Use something like http://whois.domaintools.com/ or any other HTTP 'whois' service and scrape the results

B) Use a SOCKS proxy.


See: https://unix.stackexchange.com/questions/367409/how-do-you-use-proxy-with-the-whois-command

1 Comment

Answer

'whois' runs on TCP protocol, Crawlera is an HTTP proxy so it's not going to work. You can:

A) Use something like http://whois.domaintools.com/ or any other HTTP 'whois' service and scrape the results

B) Use a SOCKS proxy.


See: https://unix.stackexchange.com/questions/367409/how-do-you-use-proxy-with-the-whois-command

Login to post a comment