1. Download the free MozBar addon for Firefox. It’s SEO stuff for spam-munching webmasters, but it’s free and it works. Note that the CSV export feature is only present in the Firefox toolbar. Not the Google Chrome version.

2. Temporarily turn off any Firefox addons you might have for modifying the appearance of Google Search results, such as GoogleMonkeyR.

3. Go to Google Search, go to Search Settings, and turn on Google Instant if you have it disabled. Turn the number of results to 100. Save. Now do a test search.

   No SERP Control Panel showing up? Click on the new SEOMoz toolbar (it’s sitting up near the top of your browser), click on the grey cogs, and select Google…

   The SERP Control Panel overlay should now appear over to the right of the search results. Note that you may also need to repeat this step, for each new search or page, in order to get the data cued up correctly for a fresh CSV output, if you have Google Instant turned off.

4. On the SERP control panel, click on “Export to CSV”…

Note than we can also do this with Bing and Yahoo, and perhaps others if you can make profiles for them. Possibly it might work with Google Scholar?

5. Open the resulting CSV file with Excel…

Above: Click on picture to see full-size version.

You even get the description/snippet from the search results, although prefaced with some junk — simply delete everything in front of keyword “Undo” in the relevant column, by using Sobelsoft’s Excel Remove (Delete, Replace) Text, Spaces & Characters From Cells Addin for Excel…

Also delete the columns with the SEO junk in them. You now have three clean columns: URL, title, and snippet. Use a formula to convert these to pretty linked HTML in a fourth column, or paste them into a mega-file of subject-specific results for further weeding and sorting.

None of the above is as robust or simple as the broken Google Extract Data and Text, and it’s to be hoped that Sobolsoft fixes this software soon for Windows 7 + IE9.

“Data objects that are able (and feasible) to become physical. We believe that things like three dimensional printers, scanners and such are just the first step. We believe that in the nearby future you will print your spare sparts for your vehicles. You will download your sneakers within 20 years.”

Google 3D Warehouse has of course being quietly doing something very similar for some years now. All their models are free (inc. commercial use) too, but legit. They even give you awesome software, Google SketchUp, for free to manipulate and alter the objects.

I’m guessing that to use these today one would need to blow the dust off an old Windows Vista PC with something like IE6 or IE7 installed, although the problem might be due to newer versions of Visual Basic runtimes or similar. The utilities don’t run well on Windows XP (I tried one, on an old laptop) because the GUI layouts are truncated in it, vital ‘save’ buttons are unreachable, and the software can’t be re-sized.

Among possible fallback options, none of the Google URL Harvester scripts for Greasemonkey work now. The clunky Outwit Hub still can’t seem to get past Google Search’s URL obfuscation and other clutter, making it fairly useless for the task. SEO software like URLHarvester and Scrapebox doesn’t seem to care about link titles or extracts, just raw URLs and PageRank.

Still working is the basic per-page method of using Multilinks (Firefox) or Linkclump (Chrome), and then my combinatory Excel spreadsheet.

** Update: found a new, free way to do it, that also harvests snippets.

My guess would be that one could probably do something similar with repository record feeds: use Excel to sort simple CSV records by the presence of keyword(s), then export only the relevant records as CSV, then load these into Omeka.

To get the smaller font size on the footnotes, paste this CSS into your theme’s styles CSS, probably at the foot of the font section (that worked for me). The plugin doesn’t add this CSS automatically.

How it was made: Lists from ROAR, OpenDOAR, BASE, Open Archives, and repository software providers were all URL-extracted and combined in Excel — then comma delimited on the / and the URL path trimmed back a little (not too far), returned to their normal state, de-duplicated, and the resulting list generally cleaned by hand/eye with the aid of a few very useful Sobolsoft addins for Excel. Any special OAI-PMH query/harvester URLs were excluded.

2,756 clean repository URLs remained. These were topped and tailed to turn them into hyperlinks, and all were put into a rough-and-ready on-the-fly Google Search engine.

Drawbacks: Users need to be aware that…

* the search results may sometimes default to being drawn from the main Google database, usually after the first few pages of results.

* you may see Google’s ads with results, if you don’t run AdBlock in your browser.

* the search tool is limited to what the Googlebot is able to “see”

* it may sometimes throw a tantrum and refuse to work.

* it likes quote marks. Idly typing in film noir will get a pitifully small amount of results, but “film noir” will get a hefty amount.

So it’s not perfect, but… you are able to use useful Google Search modifiers such as filetype:pdf or filetype:doc to discover full-text.

Re-usage: If anyone wants to use these URLs for a proper Google Custom Search Engine, feel free. Anyone can make their own 5,000-URL Google CSE, free.