+ Reply to Thread
Results 1 to 6 of 6

Using HTTP request to fill and submit form

Hybrid View

benishiryo Using HTTP request to fill... 12-09-2020, 08:39 AM
Kyle123 Re: Using HTTP request to... 12-09-2020, 08:54 AM
benishiryo Re: Using HTTP request to... 12-09-2020, 08:34 PM
benishiryo Re: Using HTTP request to... 12-15-2020, 09:59 AM
Kyle123 Re: Using HTTP request to... 12-15-2020, 10:02 AM
benishiryo Re: Using HTTP request to... 12-15-2020, 08:18 PM
  1. #1
    Forum Guru benishiryo's Avatar
    Join Date
    03-25-2011
    Location
    Singapore
    MS-Off Ver
    Excel 2013
    Posts
    5,156

    Using HTTP request to fill and submit form

    cross-posted in:
    https://www.reddit.com/r/vba/comment...d_submit_form/

    I watched wiseowl on webscrapping but he didn't share how i can use HTTP requests to fill and submit forms. so i googled "vba xmlhttp fill form"
    got 2 links i tried, but didn't know how to contextualize.

    Public Sub XmlHttpTutorial()
    'https://codingislove.com/http-requests-excel-vba/
    'tools -> references -> MS XML vX.0
    Dim xmlhttp As New MSXML2.ServerXMLHTTP60
    Dim myurl As String
    myurl = "http://www.linkedin.com"
    xmlhttp.Open "GET", myurl, False
    xmlhttp.send
    user = "someusername"
    Password = "somepassword"
    xmlhttp.setRequestHeader "Authorization", "Basic " + Base64Encode(user + ":" + Password)
    End Sub
    no idea what the last line is doing. don't know what is "Authorization", "Basic", "Base64Encode". i am guessing the last line shouldn't use "+".

    another is:

    Sub Post_HTTP_Form()
    'https://stackoverflow.com/questions/8798661/automate-submitting-a-post-form-that-is-on-a-website-with-vba-and-xmlhttp
    'Requires reference to "Microsoft XML, v6.0" or better. (Tools>References)
        Dim msXML As New XMLHTTP60, resp As String
        With msXML
            .Open "POST", "http://www.linkedin.com", False
            .setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
            .send "{PARAMETER}={VALUE}"
            resp = StrConv(.responseBody, vbUnicode)
        End With
        Debug.Print resp 'outputs to Immediate Window (CTRL+G to view)
        Set msXML = Nothing
    End Sub
    there were 3 curly brackets the contributor gave. one is the url, which i changed. not sure what parameter and value should be. although he didn't put a curly bracket on the RequestHeader, i reckon it is something i should change too.

    how do i also ensure it logs in successfuly since i don't see the browser physically?

    Thanks, if you have clicked on the * and added our rep.

    If you're satisfied with the answer, click Thread Tools above your first post, select "Mark your thread as Solved".

    "Contentment is not the fulfillment of what you want, but the realization of what you already have."


    Tips & Tutorials I Compiled | How to Get Quick & Good Answers

  2. #2
    Forum Guru Kyle123's Avatar
    Join Date
    03-10-2010
    Location
    Leeds
    MS-Off Ver
    365 Win 11
    Posts
    7,239

    Re: Using HTTP request to fill and submit form

    Your first example isn't anything to do with submitting a form, it's making a request to a resource that uses Basic Authentication (essentially a base64 encode your your username and password in a header). In reality, it's quite unusual these days to come across Basic Authentication unless it's a corporate site or an API.

    Basic Authentication is (usually) when you get a windows username and password popup box as opposed to putting credentials into the page itself.

    Your second example is submitting a form, the syntax is wrong directly as they're placeholders in that context. The parameters are passed in the format of name=value&name=value; however to get these you'll need to view the html of the form (though this isn't always straightforward). The easiest way of getting these is to use the developer tools in a browser (other than IE), go to the network tab and click on the actual POST request - if you look at the body section, you can choose to see url encoded - which is probably what you want.

  3. #3
    Forum Guru benishiryo's Avatar
    Join Date
    03-25-2011
    Location
    Singapore
    MS-Off Ver
    Excel 2013
    Posts
    5,156

    Re: Using HTTP request to fill and submit form

    thanks, Kyle. not familiar with this at all, so i'm not sure i understood.

    the website is www.linkedin.com.

    i right-clicked the username portion -> Inspect.
    chose the network tab
    don't know what you mean by actual POST request. this is what i got.
    Attachment 707976
    hence, i don't know about the body section and url encoded as well.

    after the above, how do i ensure it logs in successfuly since i don't see the browser physically?

  4. #4
    Forum Guru benishiryo's Avatar
    Join Date
    03-25-2011
    Location
    Singapore
    MS-Off Ver
    Excel 2013
    Posts
    5,156

    Re: Using HTTP request to fill and submit form

    bump

    anyone know this?

  5. #5
    Forum Guru Kyle123's Avatar
    Join Date
    03-10-2010
    Location
    Leeds
    MS-Off Ver
    365 Win 11
    Posts
    7,239

    Re: Using HTTP request to fill and submit form

    Hi, I can't open your attachment.

    Is the site you're wanting to scrape actually LinkedIn, or are you using this as an example?

    I'm asking as LinkedIn work very hard to prevent this kind of thing and are one of the few companies who have actually pursued court action against people they catch as it's in breach of their terms and conditions. Additionally, it's not the absolute hardest site to scrape but it's up there due to the way it's built, especially if you don't have a working knowledge of http requests and an understanding about how the code is organised.

  6. #6
    Forum Guru benishiryo's Avatar
    Join Date
    03-25-2011
    Location
    Singapore
    MS-Off Ver
    Excel 2013
    Posts
    5,156

    Re: Using HTTP request to fill and submit form

    here's the image again:
    https://imgur.com/BTWRcQR

    yeah, it is LinkedIn. i was actually mostly successful using selenium, but that little bit i couldn't achieve haunts me.

    hmm not sure if this is the latest news:
    https://www.forbes.com/sites/emmawoo...h=70a6f97d1b54

    regardless, you can give me an eg of a website you can fill up a username, password, and submit. maybe even excelforum?

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] VBA HTTP request not updating
    By malcmail in forum Excel Programming / VBA / Macros
    Replies: 6
    Last Post: 12-14-2020, 07:23 AM
  2. HTTP request in VBA
    By leslietsz in forum Excel Programming / VBA / Macros
    Replies: 36
    Last Post: 12-10-2016, 11:10 AM
  3. Wait for winhttp request to be done (much like XML http request)
    By JasperD in forum Excel Programming / VBA / Macros
    Replies: 21
    Last Post: 07-23-2015, 09:26 AM
  4. VBA HTTP GET request
    By WAW in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 11-21-2012, 05:08 PM
  5. Replies: 1
    Last Post: 02-05-2012, 09:31 PM
  6. Http Request Repsonse Query
    By Killavirus in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 08-25-2010, 07:38 PM
  7. Sending HTTP request with parameter.
    By gajendra.gupta in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-18-2007, 04:45 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1