+ Reply to Thread
Results 1 to 5 of 5

Web Scraping Data From A Website Where Hyperlink Does Not Change

  1. #1
    Forum Expert NeedForExcel's Avatar
    Join Date
    03-16-2013
    Location
    Pune, India
    MS-Off Ver
    Excel 2016:2019, MS 365
    Posts
    3,879

    Web Scraping Data From A Website Where Hyperlink Does Not Change

    Hi,

    I am trying to Web Scrape Data From A Website Where The Hyperlink Does Not Change.

    What that means is, there is a list of products one below another, and when any of the products is clicked, on the same page various information of the clicked product is populated on the same web page at the left side.

    What I want the code to do is, loop through all the products and get the relevant information.

    The problem here is, I cannot share the web link and I am sure without seeing the link it might be tuff to answer.

    However, any guidance toward the right direction is much appreciated as I hvn't done web scraping ever by myself..
    Cheers!
    Deep Dave

  2. #2
    Forum Contributor
    Join Date
    06-11-2014
    MS-Off Ver
    Office 2003, 2007 & 2010
    Posts
    119

    Re: Web Scraping Data From A Website Where Hyperlink Does Not Change

    Depends entriely how the web page is loaded, for example, if the structure is built into the original HTML, you're golden and can just rip the html out form the site, but if the content is dynamically loaded then you'll have a hard time.

    To scrape the html use the following (shamelessly stolen from http://www.wiseowl.co.uk/blog/s393/s...ite-html.htm):

    Please Login or Register  to view this content.
    Now that you have the HTML as a string, you can manipulate it to get the data you need. As stated before though, this only works if the html content is not created on load (as quite a few websites are starting to do to prevent this sort of manipulation)

  3. #3
    Forum Expert NeedForExcel's Avatar
    Join Date
    03-16-2013
    Location
    Pune, India
    MS-Off Ver
    Excel 2016:2019, MS 365
    Posts
    3,879

    Re: Web Scraping Data From A Website Where Hyperlink Does Not Change

    Hi,

    Thank you for the reply..

    But I don't see a loop that can loop through all the products..

    If there is any more info I can provide you with please let me know.

    If you wish, I can paste a small part of the HTML, If that is going to help.. Let me know

  4. #4
    Forum Expert JasperD's Avatar
    Join Date
    05-07-2013
    Location
    Netherlands
    MS-Off Ver
    Excel 2016
    Posts
    1,393

    Re: Web Scraping Data From A Website Where Hyperlink Does Not Change

    Use WinHTTP rather than IE automation.

    It's going to be impossible for me to help thoroughly without seeing the actual link, but you should probably look at the site traffic to see what exactly gets called when you click one of the products.
    Could be two things

    a) The site pulls the information from another (internal) link quickly either in json, table or jscript form
    b) The site actually has all the data in it already, but hidden in tables and showing on click.

    Either of them are relatively easy to handle generally, but as said, it really depends on what's the ACTUAL link.
    Feel free to send me a private message with more info.

  5. #5
    Forum Expert NeedForExcel's Avatar
    Join Date
    03-16-2013
    Location
    Pune, India
    MS-Off Ver
    Excel 2016:2019, MS 365
    Posts
    3,879

    Re: Web Scraping Data From A Website Where Hyperlink Does Not Change

    I have hopes from you or Kyle.. Coz I'v seen both of ya'll do wonders with such Web Scraping Problems.

    If I show a part of the HTML, will that help you?

    I am not at all familiar with WinHTTP or other WebScraping Techniques, hence I am unable to proceed further on my own..

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Data Scraping from the WebSite
    By sathis in forum Excel Programming / VBA / Macros
    Replies: 11
    Last Post: 07-26-2015, 10:46 AM
  2. problem while scraping data from website
    By dps700 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 12-09-2014, 09:46 AM
  3. Automated data scraping from website into excel
    By kavin in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 09-10-2014, 01:42 PM
  4. Replies: 1
    Last Post: 02-17-2014, 11:21 PM
  5. Scraping website data when Get External Data from Web doesnt work
    By Zipping2010 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 02-01-2013, 12:16 AM
  6. scraping data from a website
    By redpanda in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 07-05-2012, 01:20 PM
  7. Scraping website data into Excel
    By oliver79 in forum Excel General
    Replies: 0
    Last Post: 06-13-2010, 02:15 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1