Hi, I am trying to access website carmax.com to scrape some vehicle details and was going to use Splash for it, so i got open source version , installed it and try to get data, same as curl request i am getting on bot protection even with Splash. So I was wondering if your hosted version are more advanced so I can get access to that site with it ?
Hi, I am trying to access website carmax.com to scrape some vehicle details and was going to use Splash for it, so i got open source version , installed it and try to get data, same as curl request i am getting on bot protection even with Splash. So I was wondering if your hosted version are more advanced so I can get access to that site with it ?
0 Votes
nestor posted over 6 years ago Admin Best Answer
Hosted versions are exactly the same. You should consider using a proxy service like Crawlera if you're having issues with bot protection. You can implement Crawlera in your Splash using a Lua script, see: https://support.scrapinghub.com/support/solutions/articles/22000188428-using-crawlera-with-splash-scrapy and https://support.scrapinghub.com/support/solutions/articles/22000203566-using-crawlera-with-splash-python-requests-library.
0 Votes
1 Comments
nestor posted over 6 years ago Admin Answer
Hosted versions are exactly the same. You should consider using a proxy service like Crawlera if you're having issues with bot protection. You can implement Crawlera in your Splash using a Lua script, see: https://support.scrapinghub.com/support/solutions/articles/22000188428-using-crawlera-with-splash-scrapy and https://support.scrapinghub.com/support/solutions/articles/22000203566-using-crawlera-with-splash-python-requests-library.
0 Votes
Login to post a comment