Scrap data from Amazon using Selenium and VBA

[leprince2007]

Hello everybody,
Could you help me in my question ,please?
The code in my file searches amazon site for a product and gets the following data "name-price-link" in order to find the cheapest offer.The code gets data of first search page only.
Note:in order to make the code work without problems,you should close chrome first.
The question:
How to make the code navigate to other pages in Amazon to get data of products from?
please help me

[leprince2007]

Bump...Bump
Please help me

[leprince2007]

Bump..Bump
Is there anything unclear?
Please help me

[mogul]

If you search for Apple iPhone 11 you get a whole page of phones.

[leprince2007]

mogul, Thanks for your reply.If you search for Apple iPhone 11, you get multiple pages not one.
The code get data for the first page only in Amazon site.I want to navigate to other pages(2,3,4,etc....) in order to get data from,how?

[mogul]

I assume you are talking about the carousel down below.

Attachment 778325

In order to get the carousel you'll need to use Xpath to get elements. For example this gets you the elements in the carousel.

.//ol[contains(@class, 'a-carousel')]

to get each element you'll need to use

(.//li[contains(@class, 'a-carousel-card')])[1]

To get to next page use

.//span[text()='Next page']

If I have time I'll try to do a demo.

[leprince2007]

I tried this but didn`t work:

Set pager = ch.FindElementByXPath("//span[text()='Next page']")
         pager.Click

Please apply your solution on the file and attach it.
Help me, please.

[mogul]

Sorry, there are 2 elements in page. Try

(.//span[text()='Next page'])[1]

You need to test Xpath on page to make sure it works. I use Try Xpath in Firefox. I'm sure Chrome has some extensions.

[leprince2007]

mogul,I tried your suggestion but didn`t work.
And I tried the xpath and worked for second page only and for other pages doesnot work,because the xpath for next button change on every page:

Dim pp As Selenium.WebElements
Set pp = ch.FindElementByXPath("/html/body/div[1]/div[1]/div[1]/div[1]/div/span[3]/div[2]/div[51]/div/div/span/a[3]")
   pp.Click

Please apply your solution on the file and attach it because I`am not familiar with selenium .

[mogul]

It didn't work for me either. Scraping Amazon is difficult. I would try a Selenium forum. I tried to scrape Amazon for upcoming book releases. It is very difficult to do and Amazon constantly changes. I ended up using Barnes and Noble

[leprince2007]

After alot of search and tries,I could solve it

For j = 2 To 20
     Text = "a[aria-label='Go to next page, page " & j & "']"
     If Not ch.IsElementPresent(FindBy.Css(Text), 1000) Then
        GoTo err:
        Else
        ch.FindElementByCss(Text).Click
      End If
Next j