Making use of curb, a Ruby binding for libcurl:
require 'curb' url = "https://twitter.com" proxy = "proxy.zyte.com:8010" proxy_auth = "<CRAWLERA_APIKEY>:" c = Curl::Easy.new(url) do |curl| curl.proxypwd = proxy_auth curl.proxy_url = proxy curl.verbose = true end c.perform puts c.body_str
Making use of typhoeus, another Ruby binding for libcurl:
require 'typhoeus' url = "https://twitter.com" proxy_host = "proxy.zyte.com:8010" proxy_auth = "<CRAWLERA_APIKEY>:" request = Typhoeus::Request.new( url, method: :get, proxy: proxy_host, proxyuserpwd: proxy_auth, headers: {"X-Crawlera-Timeout" => "60000"} ) request.run print "Response Code: " puts request.response.code print "Response Time: " puts request.response.total_time print "Response Headers: " puts request.response.headers print "Response Body: " puts request.response.body
Making use of mechanize, a Ruby library for automated web interaction: Don’t forget to load the Certificate file zyte-proxy-ca.crt,
and set it using the env variable export SSL_CERT_FILE=/path/to/zyte-proxy-ca.crt
require 'rubygems' require 'mechanize' url = "https://twitter.com" proxy_host = "proxy.zyte.com" proxy_api_key = "<CRAWLERA_APIKEY>" agent = Mechanize.new agent.set_proxy proxy_host, 8010, proxy_api_key, '' res = agent.get(url) puts res.body