What are the fastest and best URL text only extractors out there?

[Dimple777]

I have an excel spreadsheet that has to make a ton of web queries (around 9000 at the moment) and is using 2 websites to filter the result page to text only before searching for the data (if you are reading this, thanks again JasperD for your help)
Those 2 text extractors are boilerpipe (http://boilerpipe-web.appspot.com/ex...ctImages=&url=) and google (http://www.google.com/gwt/x?noimg=1&u=).

Boilerpipe is super fast in returning the result but the issue is that it has a low daily usage cap and stops working after only a short while. Google (as expected) is more robust and doesn't seem to have such a limit, but it's roughly 3 times slower to return a result, and when you compound that over 9000 queries, the difference becomes very substantial.

Does anyone know better text extractors out there that will split out a text only version of a URL you give them, doing it fast (hopefully as fast/faster than boilerpipe) and do not have a daily usage limit? (or at least one that's way higher than my needs). If you do, please post away, the help would be much appreciated.

[Dimple777]

Anyone knows?

[Kyle123]

What are you doing with the data once you've got it? Are you then extracting parts or are you just saving it all in a sheet?

[Dimple777]

I'm extracting a part of it which shows how many results were returned by the URL query.

[Kyle123]

If you are only doing that, why can't you take it straight from the source of the page? Why do you need to use an intermediate HTML to Text type service?

[Dimple777]

I'm not sure how I would go around doing that. I'm still quite a newbie at excel and programming.

Here is the original thread with what I was trying to accomplish if you are interested:
http://www.excelforum.com/excel-prog...ry-solved.html

I have made a few modifications to the code that JasperD provided me. As it sits now, it has a full column of URLs, it checks each other sequentially with boilerpipe, if boiler pipe fails to result a result, it sends the query to google, then moves on to the next cell.

Here is an example of a query which would return only the result 328 (at the time of me posting this, the result will vary with time):
http://na.finalfantasyxiv.com/lodest...lang=ja&order=

And here is an example which would return a result of 0 in my excel sheet (again, at the time of posting this):
http://na.finalfantasyxiv.com/lodest...lang=fr&order=

[Dimple777]

bump to find the best text extractors out there