Scrape the site before you go home tonight
June 3rd, 2005 by Tom Johnson

Nils Mulvad, one
of the early champions of analytic journalism in Europe and founder of the Danish Institute for Analytic Reporting, demo-ed
a fast web-scrapping tool at the IRE conference this week.  Web-scrapping?  It’s a way to get just the data you need from a web site that has
a dynamic search engine.  The FECinfo site is an example: the user enters
the search terms and the site’s server returns the desired results. 

As a one-off, that works OK.  But what if you need all the data on the
server?  Turn to “RoboSuite.”  It’s a point-and-shoot, build-your-own-script
application.  A good PERL coder can do
the same thing, of course, but if you can afford it, RoboSuite is a fast
solution to data harvesting.

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

»  Substance:WordPress   »  Style:Ahren Ahimsa