Nils Mulvad, one
of the early champions of analytic journalism in Europe and founder of the Danish Institute for Analytic Reporting, demo-ed
a fast web-scrapping tool at the IRE conference this week. Web-scrapping? It’s a way to get just the data you need from a web site that has
a dynamic search engine. The FECinfo site is an example: the user enters
the search terms and the site’s server returns the desired results.
As a one-off, that works OK. But what if you need all the data on the
server? Turn to “RoboSuite.” It’s a point-and-shoot, build-your-own-script
application. A good PERL coder can do
the same thing, of course, but if you can afford it, RoboSuite is a fast
solution to data harvesting.