Inconsistant Scraping on Amazon Category pages

adcasey

Trying to scrape an amazon.co.uk category page that has two filters applied to it (Brand + Seller) which should give me a list of all my products listed on Amazon.

I want to scrape the following data:

Product Title

Rating

Total Reviews

I have tested the code and re-wrote a number of times but there I always seem to have some of the content showing somthing else as if it has skipped to another page element. i..e instead of the rating's (4.6 out of 5 stars) it changes to Prime and on these items the Total Reviews data field is completely missing.

The strange thing is it's not all product, just a handful that seem to be effected?

1 Comment

a

adcasey

said
26 days ago

Testing this some more I think it comes down to the 'Selection mode' being set to 'Automatic' and the target html blocks being slightly dynamic in the amount of content they contain.

I have tried changing the Selection Mode to CSS selector but again this causes issues as some blocks have different class names, and I believe that the XPath selector is only available for paid accounts?