An intriguing op-ed in The Washington Post on Saturday (June 20, 2009) claimed to spot fraud in the Iran elections by applying some analytic methods basically drawn from Benford's Law.  Yes, read the article, but be sure to drill down into the 140+ comments.  Most quite cogent and well argued.

The Devil Is in the Digits

Since the declaration of Mahmoud Ahmadinejad's landslide victory in Iran's presidential election, accusations of fraud have swelled. Against expectations from pollsters and pundits alike, Ahmadinejad did surprisingly well in urban areas, including Tehran — where he is thought to be highly unpopu…By Bernd Beber and Alexandra Scacco

Discovered a new, online resource for teaching spatial thinking today while attending the UCGIS Summer Assembly here in Santa Fe. Take a lookat teachspatial.org:

About TeachSpatial

TeachSpatial.org implements suggestions from a multi-disciplinary Symposium on a Curriculum for Spatial Thinking. The symposium, organized by Diana Sinton, Mike Goodchild, and Don Janelle, was hosted by the University of Redlands in June 2008. Its purpose was to discuss the merits and content of a general curriculum course on spatial thinking. One of its recommendations was to establish a wiki site to promote the discussion and sharing of resources among instructors.

Participants in the Redlands meeting were Kate Beard-Tisdale (Spatial Information Science Engineering, Maine), Marcia Castro (Global Health and Population, Harvard), Jeremy Crampton (Geosciences, Georgia State), Phil Gersmehl, Geography, CUNY Hunter), Mike Goodchild and Don Janelle (spatial@ucsb), John Kantner (School of Advanced Field Studies, Santa Fe), Steve Marshak (Geology, Illinois Urbana-Champaign), Jo-Beth Mertens (Economics, Hobart and William Smith), and Diana Sinton (Spatial Curriculum, Redlands).

What you can do here

• Create an account and contribute. Account setup is automated and fast and your email address is kept private.
• Once logged in, you can subscribe to content types (blogs, links, discussions, etc.) to get emails announcing new postings — do this from your My Account page
• From the “Create Content” page you can post:
  • schemas (e.g., models and representations) to help link concepts into broader frameworks of spatial reasoning
  • teaching resources (syllabi, lesson plans, exercises, examples of student work, etc.)
  • links of interest to this community

Jorge Camoes is one of the serious folks when it comes to dataviz. Here's some work he's done recently in U.S. unemployment data. Note especially the good state-by-state dashboard. It quickly shows New Mexico is hangin' in there.

Here are two ways to display a relatively large dataset, montly unemployment rates by state since 1976. The first one is perfect to see the overall patterns, the range from the lowest to the highest, the outliers and the slopes. An interactive version would allow the user to highlight specific series.

A small-multiple version allows the user to focus on specific states, compare them to the normal band, etc. States are ranked by labor force size and, as you can see, in the first row seven out of ten are above the US average in April. In the last row, only one is above the US average. You can also see that Michigan was not well (unemployment-wire) long before the current crisis, or a spike in Luisiana (Katrina). It pays to study this chart carefully.

Bottom line: try to see the same data from different angles. There will always be semething interesting to find.

What do you think? How would you improve these charts? Would you use a different display? Share it in the comments! (here is the data file)

Update: I usually stay away from Excel’s surface charts, but I’d like to add this one:

Also check Michael’s Horizon chart.

Interesting discussion of, fundamentally, how the Digital Revolution drives the flow from products to services and services to products. Ergo, touches on much of what is at the core of SFComplex. See….
The New Negroponte Switch — “Designing things that think they are services, and services that think they are things”. Matt Jones presentation gushing with great ideas for the “Web Meets World” change. I love the evolving printed map they made for the British Council at Salone di Mobile. A five course meal with port and insulin shots for thought.

An interesting beginning for a potentially valuable and interesting tool….

“Javascript Infoviz Toolkit — Treemaps, Radial Layouts, HyperTrees/Graphs, SpaceTree-like Layouts, and more.in this Javascript suite for building data pretties. Higher-level than processing.js. (via O'Reilly Radar and chrisblizzard on Twitter)

• Multiple Data Representations
  
  Treemaps, Radial Layouts, HyperTrees/Graphs, SpaceTree-like Layouts, and more…
• Major Browsers Support
  
  IE6+, Firefox2+, Safari3+, Opera9.5+
• Open Source
  
  Licensed under the BSD License
• Library Agnostic
  
  You may use the JIT with your favorite DOM manipulation framework
• Extensible
  
  All visualization classes are mutable, so you can easily add/override any method you want.
• Composable
  
  Visualizations can be combined in order to create new visualization methods.”

  ---

Nathan, the chap who curates the valuable blog Flowing Data, offers up a bit of hope for journalists who are worried about their employment futures and yet have invested in learning methods of data analysis.  When thinking about re-inventing ourselves, consider the phrase “data scientist.”

Rise of the Data Scientist

Photo by majamarko

As we've all read by now, Google's chief economist Hal Varian commented in January that the next sexy job in the next 10 years would be statisticians. Obviously, I whole-heartedly agree. Heck, I'd go a step further and say they're sexy now – mentally and physically.

However, if you went on to read the rest of Varian's interview, you'd know that by statisticians, he actually meant it as a general title for someone who is able to extract information from large datasets and then present something of use to non-data experts.

Sexy Skills of Data Geeks

As a follow up to Varian's now-popular quote among data fans, Michael Discroll of Dataspora, discusses the three sexy skills of data geeks. I won't rehash the post, but here are the three skills that Michael highlights:

1. Statistics – traditional analysis you're used to thinking about
2. Data Munging – parsing, scraping, and formatting data
3. Visualization – graphs, tools, etc.

Oh, but there's more…

These skills actually fit tightly with Ben Fry's dissertation on Computational Information Design (2004). However, Fry takes it a step further and argues for an entirely new field that combines the skills and talents from often disjoint areas of expertise:

1. Computer Science – acquire and parse data
2. Mathematics, Statistics, & Data Mining – filter and mine
3. Graphic Design – represent and refine
4. Infovis and Human-Computer Interaction (HCI) – interaction

And after two years of highlighting visualization on FlowingData, it seems collaborations between the fields are growing more common, but more importantly, computational information design edges closer to reality. We're seeing data scientists – people who can do it all – emerge from the rest of the pack.

Advantages of the Data Scientist

Think about all the visualization stuff you've been most impressed with or the groups that always seem to put out the best work. Martin Wattenberg. Stamen Design. Jonathan Harris. Golan Levin. Sep Kamvar. Why is their work always of such high quality? Because they're not just students of computer science, math, statistics, or graphic design.

They have a combination of skills that not just makes independent work easier and quicker; it makes collaboration more exciting and opens up possibilities in what can be done. Oftentimes, visualization projects are disjoint processes and involve a lot of waiting. Maybe a statistician is waiting for data from a computer scientist; or a graphic designer is waiting for results from an analyst; or an HCI specialist is waiting for layouts from a graphic designer.

Let's say you have several data scientists working together though. There's going to be less waiting and the communication gaps between the fields are tightened.

How often have we seen a visualization tool that held an excellent concept and looked great on paper but lacked the touch of HCI, which made it hard to use and in turn no one gave it a chance? How many important (and interesting) analyses have we missed because certain ideas could not be communicated clearly? The data scientist can solve your troubles.

An Application

This need for data scientists is quite evident in business applications where educated decisions need to be made swiftly. A delayed decision could mean lost opportunity and profit. Terabytes of data are coming in whether it be from websites or from sales across the country, but in an area where Excel is the tool of choice (or force), there are limitations, hence all the tools, applications, and consultancies to help out. This of course applies to areas outside of business as well.

Learn and Prosper

Even if you're not into visualization, you're going to need at least a subset of the skills that Fry highlights if you want to seriously mess with data. Statisticians should know APIs, databases, and how to scrape data; designers should learn to do things programmatically; and computer scientists should know how to analyze and find meaning in data.

Basically, the more you learn, the more you can do, and the higher in demand you will be as the amount of data grows and the more people want to make use of it.

 Good magazine often produces informative and innovative infographics.  Eighty of those works are now on Flickr.  Be sure to drill down into the thumbnails to see the work in detail.  Go to: http://www.flickr.com/photos/goodmagazine/sets/72157618896371005/

GOOD Transparencies Archive Thumbnails Detail 2 comments

We've been noticing since the first of the year the results of some very creative and sometime brilliant aggregation sites. (Do we need a new phrase for this format?)  These sites are richer than Google mash-ups in that they allow far more control by the user.  Some, like Mint.com or TripIt.com, also require various degrees of data entry by the user, sometimes with with a surprising degree of detail, both personal and specific.   Mapumental, below, pushes the limits of this evolution.

mySociety blog » Say hello to Mapumental

By Tom Steinberg on Monday, June 1st, 2009