Because amazon is changing the html through JavaScript, if you run view(response) in scrapy shell and then inspect element you will see class name changed to pagnNext but as it has an id so this will work response.xpath('//*[@id="pagnNextLink"]/@href').extract_first()
0 Votes
People who like this
Delete Comment
This post will be deleted permanently. Are you sure?
Hi,
I am not able to extract the XPath for pagination ('next' button) link in - https://www.amazon.com.au/s?k=tablet
I tried in scrapy shell to extract:
response.xpath('//li[@class="a-last"]/a/@href').extract_first()
but it doesn't output the link.
Please see the attached screenshot of the HTML code.
Many thanks,
Vinay
Attachments (1)
htmlcode.png
31.1 KB
0 Votes
1 Comments
aurish_hammad_hafeez posted over 5 years ago Admin
Because amazon is changing the html through JavaScript, if you run view(response) in scrapy shell and then inspect element you will see class name changed to pagnNext but as it has an id so this will work
response.xpath('//*[@id="pagnNextLink"]/@href').extract_first()
0 Votes