Using HTTP request to fill and submit form

**benishiryo** · 12-09-2020, 08:39 AM

cross-posted in:
https://www.reddit.com/r/vba/comment...d_submit_form/

I watched wiseowl on webscrapping but he didn't share how i can use HTTP requests to fill and submit forms. so i googled "vba xmlhttp fill form"
got 2 links i tried, but didn't know how to contextualize.

Please Login or Register  to view this content.

no idea what the last line is doing. don't know what is "Authorization", "Basic", "Base64Encode". i am guessing the last line shouldn't use "+".

another is:

Please Login or Register  to view this content.

there were 3 curly brackets the contributor gave. one is the url, which i changed. not sure what parameter and value should be. although he didn't put a curly bracket on the RequestHeader, i reckon it is something i should change too.

how do i also ensure it logs in successfuly since i don't see the browser physically?

**Kyle123** · 12-09-2020, 08:54 AM

Your first example isn't anything to do with submitting a form, it's making a request to a resource that uses Basic Authentication (essentially a base64 encode your your username and password in a header). In reality, it's quite unusual these days to come across Basic Authentication unless it's a corporate site or an API.

Basic Authentication is (usually) when you get a windows username and password popup box as opposed to putting credentials into the page itself.

Your second example is submitting a form, the syntax is wrong directly as they're placeholders in that context. The parameters are passed in the format of name=value&name=value; however to get these you'll need to view the html of the form (though this isn't always straightforward). The easiest way of getting these is to use the developer tools in a browser (other than IE), go to the network tab and click on the actual POST request - if you look at the body section, you can choose to see url encoded - which is probably what you want.

**benishiryo** · 12-09-2020, 08:34 PM

thanks, Kyle. not familiar with this at all, so i'm not sure i understood.

the website is www.linkedin.com.

i right-clicked the username portion -> Inspect.
chose the network tab
don't know what you mean by actual POST request. this is what i got.
Attachment 707976
hence, i don't know about the body section and url encoded as well.

after the above, how do i ensure it logs in successfuly since i don't see the browser physically?

**benishiryo** · 12-15-2020, 09:59 AM

bump

anyone know this?

**Kyle123** · 12-15-2020, 10:02 AM

Hi, I can't open your attachment.

Is the site you're wanting to scrape actually LinkedIn, or are you using this as an example?

I'm asking as LinkedIn work very hard to prevent this kind of thing and are one of the few companies who have actually pursued court action against people they catch as it's in breach of their terms and conditions. Additionally, it's not the absolute hardest site to scrape but it's up there due to the way it's built, especially if you don't have a working knowledge of http requests and an understanding about how the code is organised.

**benishiryo** · 12-15-2020, 08:18 PM

here's the image again:
https://imgur.com/BTWRcQR

yeah, it is LinkedIn. i was actually mostly successful using selenium, but that little bit i couldn't achieve haunts me.

hmm not sure if this is the latest news:
https://www.forbes.com/sites/emmawoo...h=70a6f97d1b54

regardless, you can give me an eg of a website you can fill up a username, password, and submit. maybe even excelforum?

Using HTTP request to fill and submit form

LinkBack

Thread Tools

Rate This Thread

Display

Using HTTP request to fill and submit form

Re: Using HTTP request to fill and submit form

Re: Using HTTP request to fill and submit form

Re: Using HTTP request to fill and submit form

Re: Using HTTP request to fill and submit form

Re: Using HTTP request to fill and submit form

Thread Information

Users Browsing this Thread

Similar Threads

[SOLVED] VBA HTTP request not updating

HTTP request in VBA

Wait for winhttp request to be done (much like XML http request)

VBA HTTP GET request

Custom Form with Submit Button, Email and then Submit to Database

Http Request Repsonse Query

Sending HTTP request with parameter.

Bookmarks

Bookmarks

Posting Permissions