How would I grab the source code from a website and place it into a text file using vba?

**111StepsAhead** · 06-14-2012, 09:55 AM

Essentially it would be a web scraper. I'm either looking for existing free code or a good place to learn how to make one.

**romperstomper** · 06-14-2012, 10:16 AM

I'm sure there are many different ways, but this may help

Sub testit()
    LogIESource "http://www.microsoft.com", "C:\microsoft.txt"
End Sub
Sub LogIESource(strURL As String, strFilePath As String)
    Dim objIE            As Object
    Dim  FileNumber as long

    Set objIE = CreateObject("InternetExplorer.Application")
    ' Get unused file

    objIE.Navigate strURL
    Do While objIE.ReadyState <> 4
        DoEvents
    Loop
    FileNumber = FreeFile
    ' Create log file
    Open strFilePath For Output As #FileNumber

    Print #FileNumber, objIE.Document.DocumentElement.outerhtml

    ' Close file.
    Close #FileNumber
End Sub

**Kyle123** · 06-14-2012, 10:16 AM

Typically you wouldn't use vba for this, however since you want t ouse VBA, see if this gets you started:

Sub GetSource()
    With CreateObject("MSXML2.XMLHTTP")
        .Open "GET", "http://www.excelforum.com"
        .send
        Do: DoEvents: Loop Until .ReadyState = 4
        
        Open ThisWorkbook.Path & "\Output.txt" For Output As #1
            Print #1, .responsetext
        Close #1
        
    End With
End Sub

Webscraping can get very complex very quickly and as I said VBA isn't the ideal tool

**111StepsAhead** · 06-14-2012, 10:23 AM

Thanks for the quick responses. Hope you guys didn't have to put too much effort into making that code. I am trying to expand on what I've learned over the last few months here and this seemed like a fun way to do that. Both of your guys code worked great.

I will mark this thread as solved but as an extra question, what programming language would be better for webscraping?

**Kyle123** · 06-14-2012, 10:25 AM

Probably Python

**111StepsAhead** · 06-14-2012, 10:31 AM

Originally Posted by Kyle123

Probably Python

I shall keep this in mind when I get some free time.

How would I grab the source code from a website and place it into a text file using vba?

LinkBack

Thread Tools

Rate This Thread

Display

Hybrid View

How would I grab the source code from a website and place it into a text file using vba?

Re: How would I grab the source code from a website and place it into a text file using vb

Re: How would I grab the source code from a website and place it into a text file using vb

Re: How would I grab the source code from a website and place it into a text file using vb

Re: How would I grab the source code from a website and place it into a text file using vb

Re: How would I grab the source code from a website and place it into a text file using vb

Thread Information

Users Browsing this Thread

Bookmarks

Bookmarks

Posting Permissions