I found Tony Hirst’s UK University Autodiscoverable RSS feeds service on ScraperWiki, while looking for a means to pull RSS feeds from the 3,000 links in the JURN Directory. Tony’s hosted script on ScraperWiki (it’s like a supercharged British version of Yahoo Tubes) pulls in the URLs of 137 British universities, then autodetects all the RSS feeds on those URLs, and also searches the html for links to .rss, .rdf, and similar feed-indicating links.

You can download the resulting table as a .csv file for Excel, and in the “furl” column you find all the feed links. Cart away all the dead from the list, and you have around 85 autodiscoverable news feeds which validate and show as active in Feeddemon. And which are not about “clearing” course places for students, or library minutiae aimed at students and staff.

It’s a useful starting point. But I then had to manually add in nearly all the Midlands universities — Birmingham; Keele; BCU and BIAD; Coventry; Staffordshire; Nottingham; MMU at Crewe — since the autodetect hadn’t worked for any of them. Yet they all have feeds. Sadly no RSS news is offered by Wolverhampton or Worcester.

So I think this shows the limitations of RSS autodiscovery, even when it’s running from a good data source and using an accomplished bit of scripting. And possibly it also shows the limitations of scraping and bots in general. Rather than spend the time learning to script / setting up the script, in this case it’d probably be easier and cheaper to pay someone in India £50 to run through all the 137 UK universities home-page URLs and manually discover each RSS feed for news / jobs / events and more.

If you’re interested, here’s my final checked and Midlands-augmented OPML file for plugging into your feedreader. Just knock “.txt” off the end of the filename uk_unis_rss_feeds_jan_2013.opml.txt to get the .opml file for import. If you augment it with missing university feeds from your own region of the UK, please post a link to your OPML in the comments on this post.

Advertisements