Science and simulation for the greater good
Nov 5th, 2006 by JTJ

A former student of colleague Steve Ross sends this interesting report on how simulation models can/are being used in the real world:

I’m the communications officer for the International Research
Institute for Climate and Society
at Columbia University. The IRI specializes
in making forecasts of climate for every part of the world by using data from
satellites, meteorological stations and proxy records (tree rings, corals, etc)
to run models. The models tell us, with varying degrees of certainty, how much
off the “norm” rainfall, temperature and humidity will be for a given place in
the world.  We’re not so much interested
in long-term climate change (global warming, sea level rise, etc) as we are in
season-to-season changes (e.g. monsoonal patterns, drought, flooding, etc.).
And the IRI isn’t a purely academic institution — its main objective is to use
the forecasts and climate monitoring to develop and undertake projects that
mitigate the effects of climate change in developing countries. We never do
this alone: all of our efforts are in collaboration with scientists, policymakers
and NGOs in these countries.

Here’s a brief description of some projects we’re currently working

*Climate and malaria:*
The IRI collects an enormous amount
of temperature, rainfall and humidity data for southern Africa. As it it turns
out, the presence or absence of malaria in a given region depends strongly on
these three climate factors, so scientists here developed a mapping tool that
shows the risk of a malaria epidemic for every month of the year in every part
of sub-Saharan Africa. We train health workers from countries in this area on
how to use the information to adequately prepare for epidemics. (see /

*Climate and fire-management*
Fires in Indonesia damage unique
and delicate ecosystems, increase carbon dioxide emissions, and produce noxious
smoke and haze that leads to thousands of hospitalizations every year. Since
the intensity and duration of these fires depend on the amount of rainfall the region
receives every season, the IRI is using its rainfall forecasts to develop an early-warning
system that policymakers and NGOs can use for planning purposes. For example, if
our models tell us there is a strong chance of drought conditions in the next 3-month
period there, our Indonesian partners can take specific actions, such as conserving irrigation water so that the fields where these fires
occur aren’t drained completely and therefore aren’t as susceptible to burning.
(see /

*Index-based weather insurance for farmers*
The IRI and the Commodity Risk Management Group at the World
Bank are involved in a project to develop insurance contracts that protect Malawi
farmers against periodic, crop-destroying droughts. Traditionally, farmers would
take out loans to buy seeds at the start of every season. If a drought occurred,
the farmers’ crops would die, and they wouldn’t be able to pay back the banks. But
under this new program the farmers can purchase an insurance (a very small percentage
of the price of the seeds) against crop loss when they buy seeds. If a drought occurs,
the farmers get a full or partial payout and can use the money to repay their loans.
IRI’s role in this is to use its weather monitoring data for the region to help
the local insurance companies develop reliable contracts. The fascinating aspect
of this program is that it is completely subsidized by the farmers.  (no link
available yet)

My role in all this is to make these and other projects known
to the public at large.  Many of you receiving this email are journalists of
one species or another. If you are developing stories or graphics that have to do
with climate or earth science, think of me. I’ll put you in touch with experts or
send you bucketfuls of GIS and other data.

Francesco Fiondella
Communications Officer
International Research Institute for Climate and Society
Lamont-Doherty Earth Observatory
61 Route 9W
Palisades, NY 10964
francesco [at]

English translation of Minard's classic chart of Napoleon's March
Nov 4th, 2006 by JTJ

Mike Stucka  today provided a rich set of links during a discussion of mapping on NICAR-L.  If you're interested in Minard's map/chart/info graphic of Napeoleon's march to Russia — and especially if you use it as a teaching tool — check out the links below.

Click here to see the SAS code (zoomed-in view)

Charles Minard is a famous map-creator from the late 1800's.
Perhaps his most famous map is the following showing
Napoleon's March on Moscow in 1812.

I looked at several other people's version of Minard's map
on the following webpage and devised an idea for my own
version, using SAS/Graph's "proc gmap".

I used longitude/latitude, troop number, and city data from the following webpage
in combination with the temperature data from the following translation of Minard's Map.

I'm somewhat "geographically challenged" when it comes to europe, especially
with the city names on the Minard map (and I'm probalby not alone?)
Therefore I decided to plot Napoleon's path on *modern* maps, with the
country names labeled. Also, I wanted to give a "50,000-mile" view of
the map, so people could see where this area was in relation to other
areas of europe (such as France).

* Here's my 50,000-mile map.
* Which drills-down to this Zoomed-in map.
* Here is the SAS code used to generate the maps.

Here are some details about how I created my map(s) in SAS/Graph...

For both of these maps, I used a combination of the sas maps.europe
and as the base map. I created custom/subsetted/clipped
versions of these maps using SAS' "proc gproject", and specifying
the latmax/latmin/longmax/longmin for the area I wanted in my map.

To get the blue ocean/sea water to appear only in the map area I use
an annotated rectangular polygon with corners at the exact same
long/lat coordinates as the corners of my map, and annotate it
'behind' the map. (If I had used "goptions cback=blue" that would
have filled the entire background of the page with blue, not just
the wanter area in the map.)

In the 50,000-mile map, I annotate country names at long/lat
positions of my choosing, and I annotate a dashed-line box around
the area that will be shown for the drilldown. In this dashed-box
area, I use annotate's "html" variable to encode a drilldown, so
that when you click inside this rectangle it drills-down to the
zoomed-in map. I also annotate a dot at the city locations (and
I annotate city labels in the zoomed-in map).

I use the same technique in both maps to show Napoleon's path.
I take the long/lat values, and use them as vertices for a line,
and connect the dots with line segments (using annotate move/draw
functions). The size/width of the line is calculated based on the
number of troops (ie, men) still alive during that leg of the trip.
One problem with this technique was that it produced big/jagged
gaps/transitions at the line vertices (especially where the lines
changed directions sharply). To smoothe this out, I annotated a
'pie' (filled circle) at each vertex, with the diameter of the
pie being the same as the width of the line. With this annotation
(as with most all of the lat/long-based annotation in this map)
I combine the annotation with the map, use "proc gproject" to do
the map projection, and then separate the annotate from the map
(this guarantees that things line up in the correct position).

On the zoomed-in map, notice that when you hover your mouse over the
city names, you see the city name (and additional info, if available)
for that city. If you click on the city names, that launches a
google search for information about that city (including the words
'Napoleon' and '1812' in the search).

When you hover your mouse over the vertex points of Napoleon's path,
you'll get a html charttip/flyover-text showing the number of troops
at that point during the march. At first I made these hotspots the
exact same size as the visual vertex dots, but the small ones were
too small -- therfore I annotate a larger dot 'behind' the map for
each vertex, and the charttip is based on the size of those 'invisible'
vertex points 🙂

Surprisingly the toughest part of the map was the temperature plot.
After trying a few different approaches, I decided to base the position
of the temperature dots and the axes & gridlines & labels all using
long/lat coordinates. The distance from the minimum gridline to the
maximum gridline is 1 degree of latitude (not to be confused with
degrees celcius or degrees farenheit! ;). And for the horizontal
positions, I use he position of the cities/battles corresponding to
the temperatures. I color the temperature dots an 'ice cold' light
blue, since they represent cold temperatures.

/* Written by Robert Allison ( */

Analysis tied to making the "story" visual
Nov 3rd, 2006 by JTJ

The NYTimes Michael Gordon broke a fine story on Nov. 1, 2006 — “U.S. Central Command Charts Sharp Movement of the Civil Conflict in Iraq Toward Chaos.”  The grabber in the story was a single PowerPoint slide some unnamed source slipped to Gordon that illustrated where the U.S. was on the “continuum of chaos” in Iraq. 

The story is a good and insightful read, but the important lesson to take away is how what might seem to be random events — in this case violent events — and be understood and communicated as the visual aggregate of a complex phenomenon.  Journalists might give some thought to how can we (a) visually present both static and dynamic events in our towns and (b) what do we first have to learn and understand to do so? 

Only then will we be able to ask the right and pertinent questions of the military, corporations and public officials — who are all using these and similar techniques.

A quiz for your newsroom?
Oct 26th, 2006 by JTJ

Have your colleagues and student's give this a shot.

Subject: Geography—give it a


Since the Middle
East has been in the news for a long time we should know at least where these
countries are on the map. So here's a little test of your knowledge of

Yup, that time of the decade is again fast approaching
Oct 26th, 2006 by JTJ

FYI, folks:

Cynthia Taeuber will present her online course “Using the Census's
American Community Survey (ACS)” at Nov.17 – Dec. 15.
She will be available for questions and comments on a private
discussion board throughout this period.

Prior to 2006, analysts had to make do with increasingly out-of-date
detailed information about households and individuals while they waited
for the next decennial census. Starting in 2006, this information will
be made available on an annual basis in the ACS.

This course shows what sort of information is included, how to obtain
it, and what methodological and sample size issues present themselves.

If you have not made use of similar Census data previously, learn how
you can leverage these improvements in data currency and timeliness for
your projects.  If you have used decennial census data before, you will
benefit by learning about the methodological differences between this
Survey and the decennial census long form – they affect the results and
you may make errors if you don't know how to handle the differences.

Ms. Taeuber, a senior policy advisor at the University of Baltimore's
Jacob France Institute, has 30 years of experience at the U.S. Census
Bureau, directed the analytic staff for the American Community Survey,
and received the Commerce Dept.'s Gold Medal Award for her innovative
work on the American Community Survey.  She is the author of “The
American Community Survey:  Updated Information for America's
Communities,” and more.

As with all online courses at, there are no set hours
when you must be online; we estimate you will need 7-15 hours per week.


Peter Bruce

P.S.  Also coming up:

Nov. 3 – Cluster Analysis (useful for customer segmentation)
Nov. 17 – How to deal with missing data
Nov. 27 – Basic Concepts in Probability and Statistics
612 N. Jackson St.
Arlington, VA 22201

Games: They ain't kid-stuff
Oct 25th, 2006 by JTJ

The past week or two has brought some press stories about games being designed/developed as tools for learning, as in “productive learning,” not learning how to inflict terror or be a better car-jacker.  We recently ran across the site below, “Social Impact Games.”  It's well worth a visit, as are the others.

We think these have great potential for journalism as tools to help readers/viewers learn how government, eduction, schools, the legal system and nature work. 

Good links to simulation games:

*) Social Impact Games.  This one is a very rich jump site: or




Frankly, and off the record, our favorite is the Anti-Bush game:

”The Anti-Bush Video Game”

From the website: “Combines humor, opinion, and fact to bring an
entertaining and informative video game adventure to people everywhere.

“The use of this medium will hopefully reach many people who have
not had the time or interest to read up on some of the appalling things
that have taken place in our government and society over the past four
years. For those of you who are paying attention, hopefully this game
has helped to clarify some of the important things at stake in the
upcoming elections. I realize that this game does not cover every
issue, problem, and appalling action of the Bush administration. There
are too many stories to report. Some issues ended up taking a back seat
to others. Just know that this is just a silly game and please inform
yourself for real and read books…and most importantly…please vote.”

By Starvingeyes/J. Oda.

Something less than half a measure
Oct 17th, 2006 by JTJ

A brief comment was passed along on the NICAR-L (National Institute for Computer-Assisted Reporting) listserv this morning by Daniel Lathrop, of the Seattle Post-Intelligencer.  Said he:

Really interesting story on lobbyists-related-to-lawmakers in The USA
Today. I think those of us who cover money-in-politics should all have
a little story envy on this one.

Daniel Lathrop
Seattle P-I

Well, yeah.  An interesting story, but also one demonstrating why newspapers as institutions simply do not grasp the shift in power inherent in the Digital Age, a shift away from institutions and to citizens. 

First, the story reports: “The family connections between lobbying and lawmaking are prompting
complaints that Congress is not doing enough to police itself
.”  Fair enough, but can't you SHOW us, in the online version, the evidence to support this sweeping generalization of “prompting complaints.”  Why should we take your word for it, guys, when the evidence must be at hand.

Second, “…USA TODAY reviewed thousands of pages of financial disclosures and
lobbyist registrations, property records, marriage announcements and
other public documents to identify which lawmakers and staffers had
relatives in the lobbying business.
”  WOW!  Would I like to see those pages, and even drill down into them to see if there's anything there related to my representative.  But nooooooooo.  The paper must of had some way to manage all this
public-record data, some way to cross-reference it, to search it, to retrieve documents and
content.  Why not put all that up on the
web and let readers peruse their own subjects of interest?

Ironically, an example of the power shift mentioned above turns up, buried in a sidebar to the story, “Little Accountability in Earmarks.”  There we find reference to something called the Sunlight Foundation.  I had not heard of the Sunlight Foundation, but, hey, it's only been around since the first of the year.  It turns out this organization is doing just what newspapers should be doing: leveraging the power of the digital environment to connect people to the data and tools needed to analyze that data so they can make informed decisions.

Another opportunity missed by the industry, and tragically so.

Some fine work by the students
Oct 15th, 2006 by JTJ

All we can say is that we were out of the country when this package of stories first appeared in early September, and we missed it.  (Yeah, so much of the all-the-time-anywhere of the Internet.)  But do take a look at what the graduate journalism students at Northwestern University served up.  Good research coupled with good presentations on a topic most serious and under-reported.

Northwestern University Data Dilemma

Data Dilemma: Privacy in an Age of Security

Northwestern University's News21 fellows look at America's new system
of surveillance, developed by the government with the help of private
data mining firms after 9/11. One story uncovers new details about a secretive program
in which the Education Department shared personal information on
hundreds of student loan applicants with the FBI. Two immersive
interactive presentations explore the digital trails we leave behind us in our daily lives and show you government data-mining initiatives that might incorporate information about you.

Using Flash to illustrate the Crash
Oct 12th, 2006 by JTJ

A fine piece of work on the NYTimes web site following the crash of the small plane into an East Side hi-rise apartment building on Wednesday (11 Oct. 2006).  Check out “Small Plane Hits Building in Manhattan

Teasing out attitudes from text
Oct 5th, 2006 by JTJ

Eric Lipton has a piece in Wedneday's (4 Oct. 2006) NYTimes about some “new” research efforts to come up with software “that would let the [U.S.] government monitor negative opinions of the United States or its leaders in newspapers and other publications overseas.”  (See “Software Being Developed to Monitor Opinions of U.S.“)  Surely this is an interesting problem, and one made especially difficult when the translation factor kicks in. 

This is not, however, the first attempt to gin-up such software.  We have long admired the work done some years ago at the Pacific Northwest National Laboratory in the ThemeRiver™ visualization.

It “…helps users identify time-related
patterns, trends, and relationships across a large collection of
documents. The themes in the collection are represented by a 'river'
that flows left to right through time. The river widens or narrows to
depict changes in the collective
strength of selected themes in the
underlying documents. Individual themes are represented as colored 'currents' flowing within the river. The theme currents narrow or widen
to indicate changes in individual theme strength at any point in time.
  Status: An interactive proof of concept prototype has been developed. Download a QuickTime video about ThemeRiver (20MB)

We hope the PNNL will continue by giving us more of this intriguing tool.

»  Substance:WordPress   »  Style:Ahren Ahimsa