How to get content from Web with Xpath?

**Severduf** · 05-14-2024, 08:18 PM

I have this macro with which I can search by Xpath if the HTML is simple

Sub GetByXpathFromHTML_String()
        Dim XDoc As Object, myList As Object
        Dim myStr As String
            
        Set XDoc = CreateObject("MSXML2.DOMDocument.6.0")
        XDoc.async = False
        XDoc.SetProperty "SelectionLanguage", "XPath"
        XDoc.validateOnParse = False
        
        myStr = "<html> " & _
                   "<body> " & _
                       "<div>This is a division " & _
                          "<a  target='_top' href='default.asp'>HTML Home</a>" & _
                          "<a  target='_top' href='default.asp'>HTML Introduction</a> " & _
                          "<a  target='_top' href='default.asp'>HTML Editors</a> " & _
                          "<a  target='_top' href='default.asp'>HTML Basic</a>" & _
                       "</div> " & _
                       "<span class='color_h1'> HTML Tutorial</span>" & _
                   "</body>" & _
              "</html>"
                     
        XDoc.LoadXML (myStr)
        
        XDoc.Save (ThisWorkbook.Path & "\Test.xml")
       
        Set myList = XDoc.SelectNodes("//a[contains(.,'HTML Editors')]")
        
        If XDoc.parseError.ErrorCode <> 0 Then
            MsgBox "Error at Line No: " & XDoc.parseError.Line & vbCrLf & vbCrLf & "Error : " & XDoc.parseError.reason
            GoTo SafeExit:
        End If
        
        If myList.Length = 0 Then GoTo SafeExit:
        
        MsgBox "Number of found items= " & myList.Length
        
        MsgBox "Found item: " & myList(0).Text
       
    SafeExit:
        If myList.Length = 0 Then
            MsgBox "Not found....", vbCritical
        End If
        Set myList = Nothing
        Set XDoc = Nothing
    End Sub

Now, if I want to use the same Xpath but with the source change to an URL, I get the error "DTD is prohibited".

Why is happening this and how to fix it?

I can't use 3rd party tool like Selenium since I cannot use or installed exe files due to security policy in company.

Thanks in advance for any help

Sub GetXpathFromWeb()
    Dim strURL As String, objHTTP As Object, XDoc As Object, myList As Object, Num As Integer, i As Integer
    Dim myStr As String
    strURL = "https://www.w3schools.com/html/"
    
    Set objHTTP = CreateObject("MSXML2.XMLHTTP.6.0")
    
    objHTTP.Open "GET", strURL, False
    objHTTP.send
    
    Set XDoc = CreateObject("MSXML2.DOMDocument.6.0")
    XDoc.async = False
    XDoc.SetProperty "SelectionLanguage", "XPath"
    XDoc.validateOnParse = False
       
    XDoc.LoadXML (objHTTP.responseText)
    XDoc.Save (ThisWorkbook.Path & "\Test.xml")
   
    Set myList = XDoc.SelectNodes("//a[contains(.,'HTML Editors')]")
    
    If XDoc.parseError.ErrorCode <> 0 Then
        MsgBox "Error at Line No: " & XDoc.parseError.Line & vbCrLf & vbCrLf & "Error : " & XDoc.parseError.reason
        GoTo SafeExit:
    End If
    
    If myList.Length = 0 Then GoTo SafeExit:
   
SafeExit:
    If myList.Length = 0 Then
        MsgBox "Not found....", vbCritical, objHTTP.Status
    End If
    Set myList = Nothing
    Set XDoc = Nothing
    Set objHTTP = Nothing
End Sub

**AliGW** · 05-15-2024, 10:36 AM

Administrative Note:

We would very much like to help you with your query, however it has been brought to our attention that the same query has been posted on one or more other forums and you have not told us about this. You are required to do so. Cross-posts are allowed but you must provide a link to your posts on other sites.

Please see Forum Rule #7 about cross-posting and adjust accordingly. Read this to understand why we (and other sites like us) consider this to be important: https://excelguru.ca/a-message-to-forum-cross-posters/

(Note: this requirement is not optional. No help to be offered until you provide a link or, for members with fewer than 10 posts, a comment telling us where else you have posted this query.)

**Severduf** · 05-15-2024, 02:09 PM

Hi AliGW, actually this question is a kind of different and in the other site after one day normally there is almost zero possibilities to get an answer. In that other quesion I didn´t get a solution anyway.

**AliGW** · 05-16-2024, 03:12 AM

Originally Posted by Severduf

Hi AliGW, actually this question is a kind of different and in the other site after one day normally there is almost zero possibilities to get an answer. In that other quesion I didn´t get a solution anyway.

Doesn't matter - you still need to abide by our rules. Thanks for the links - don't forget if you do this again to include them.

**Haluk** · 05-15-2024, 04:06 PM

Originally Posted by Severduf

......
Now, if I want to use the same Xpath but with the source change to an URL, I get the error "DTD is prohibited".

Why is happening this and how to fix it?

You can overcome that error by setting the below property,

    XDoc.setProperty "ProhibitDTD", False

But, you will probably will get another error on a different line and this will continue ..... As i told you on the other side, it is not a good idea to work with this kind of code on HTML pages as there may be numerous errors in the HTML source such as missing end brackets and so on. But XML parsing requires the loaded XML file to be free of errors.

**Severduf** · 05-15-2024, 07:22 PM

Hi Haluk, thanks for answer. After some research, I've tried that option yesterday and didn´t work either.

After adding that line I get the new error.

"Missing equals sign between attribute and attribute value"

Additionally, in some sites mentioned that removing the string "<!DOCTYPE html>" from the html could work, but in my case doesn´t work.

Regards

**Haluk** · 05-16-2024, 02:55 AM

Well, it is not a surprise to me.....as I told you in my post #4 you will get a new error.

It is better for you to use some other techniques.

**Severduf** · 05-16-2024, 03:11 AM

Thanks for your help

How to get content from Web with Xpath?

LinkBack

Thread Tools

Rate This Thread

Display

Hybrid View

How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Re: How to get content from Web with Xpath?

Thread Information

Users Browsing this Thread

Similar Threads

FilterXML/Xpath

Import Xml using XPATH

Web Scrape Using Xpath

Using XPath with ID, not a path

XPath.SetValue

Problem with XPath.SetValue

XML & XPATH in EXCEL

Tags for this Thread

Bookmarks

Bookmarks

Posting Permissions