Applied Complexity in Havana
Mar 28th, 2010 by Tom Johnson

In January, Team Santa Fe (Tom Johnson, Steve Guerin, Nick Bennett, and Alfredo Covaledo [Bogota]) rolled up in Havana to attend the 10th Congress of Complexity Studies in Cuba. (Steve, Nick and Alfredo also taught a day-long workshop on Netlogo and simulation modeling following the conference.)

One of the conference attendees from the Univ. of Guanajuato (Mexico) shot some footage of Steve's Simtable demos and also a lecture by Dr. Niles Eldredge. Those videos can be seen here:

-tom johnson

Pivot Tables in Excel Webcast [free}
Mar 19th, 2010 by Tom Johnson

OReilly Webcast

Pivot Tables in Microsoft Excel

Presented by:

Michael Milton, author of Head First Excel, Head First Data Analysis, and Great R: Level 1

Pivot tables let you put together in seconds data summaries that would take forever to create with Excel formulas. That speed gives you the ability to get answers about your data as quickly as you can think up questions for it. Pivot tables are one of Excel's most versatile features, but they can be tough to break into.

Even if you're new to Excel, this live presentation will get you using pivot tables like a champ, and you'll learn

  • How to group and summarize data at warp speed using pivot tables
  • How to create explore data with pivot tables and create new segments for analysis
  • How to recognize data that can and cannot be processed by pivot tables

About Michael Milton

Michael is the author of Head First Excel, Head First Data Analysis, and the forthcoming Great R: Level 1. He has spent most of his career helping nonprofit organizations improve their fundraising by interpreting and acting on the marketing data they collect from their donors.

email register
email box top
Price: Free
email box bottom
email box top
email calender icon Wednesday,

March 24th
email box
email box top
email clock icon 10:00am-

11:00am PT
email box bottom

How 'bout a new term: "NewsViz"?
Feb 27th, 2010 by Tom Johnson

 Many of us are commonly using the term “data visualization” or “dataviz” or even “infoviz.”  Perhaps we should add to the lexicon “newsviz.”  If so, you saw it here first.  Maybe.  In any event, check out this interesting page at Slate.

News Dots: The Day's Events as a Social NetworkAn interactive map of how every story in the news is related, updated daily.

Like Kevin Bacon's co-stars, topics in the news are all connected by degrees of separation. To examine how every story fits together, News Dots visualizes the most recent topics in the news as a giant social network. Subjects—represented by the circles below—are connected to one another if they appear together in at least two stories, and the size of the dot is proportional to the total number of times the subject is mentioned.

To use this interactive tool, just click on a circle to see which stories mention that topic and which other topics it connects to in the network. Double click a dot to zoom in on it. From there, you can click on any connected dot to see which stories mention both subjects. To zoom out, just double click in white space or use the zoom out button in the upper left corner. The buttons in the upper right can toggle the emphasis between the importance of a subject and how recently it has appeared on the radar. A more detailed explanation of how News Dots works is available below the graphic.

Analysis,  Feb. 24, 2010: Three potential 2012 Republican presidential nominees–Sarah Palin, Mitt Romney, and Tim Pawlenty–all cluster around the Republican Party dot. But newly minted senator Scott Brown dwarfs them all.

This is a work in progress, so please send us your ideas for features you'd like to see or other ways we can improve it.

How News Dots works

Step 1: Behind the scenes, News Dots scans all articles from major publications—about 500 stories a day—and submits them to Calais, a service from Thompson Reuters that automatically “tags” content with all the important keywords: people, places, companies, topics, and so forth. Slate's tool registers any tag that appears at least twice in a story.

Step 2: Each time two tags appear in the same story, this tool tallies a connection between them. For example, a story about a planned troop increase in Afghanistan reform might return tags for President Obama, the White House, and Afghanistan. These topics are now connected:

Step 3: As this tool scans hundreds of stories, this network grows rapidly, and “communities” begin to form among the tags. Subjects that are highly connected—those that appear together in many stories—cluster together in the network. This occurs in the same way that a picture of the social network of your Facebook friends would reveal clusters of friends from high school, college, and work, with some unexpected connections between them when friends belong to multiple cliques.

Step 4: The news network that results is visualized using Slate's custom News Dots tool, which is built using an open-source Actionscript library called Flare. Tags are displayed if they appear in at least four stories, and connections are made if at least two stories link those two subjects. The visualization covers the previous three days of news and is updated daily.


More Visualization Links on Twitter
Jan 23rd, 2010 by Tom Johnson

Thanks to Steve Doig for the pointer to….

More Visualization Links on Twitter

By: Jeff Clark    Date: Sat, 23 Jan 2010

In a recent post I showed the Top 20 Individual Data Visualizations Mentioned on Twitter and remarked that many of the most frequently mentioned twitter links were to collections of visualizations. Shown below is a meta list of the top collection-type data visualization or infographic links.

Top Collections of Data Visualization Links

  1. 50 Great Examples of Data Visualization – Webdesigner Depot

  2. Data Visualization and Infographics Resources – Smashing Magazine

  3. 15 Stunning Examples of Data Visualization – Web Design Ledger

  4. 20 Essential Infographics & Data Visualization Blogs – Inspired Magazine

  5. Is Information Visualization the Next Frontier for Design? – Fast Company

  6. 28 Rich Data Visualization Tools – InsideRIA

  7. The Beauty of Infographics and Data Visualization – Abduzeedo

  8. 50 Great Examples of Data Visualization – Sun Yat-Sen University

  9. 20 Inspiring Uses of Data Visualization – SingleFunction

  10. 5 Best Data Visualization Projects of the Year – 2009 – FlowingData

  11. Data Visualization: Stories for the Information Age – BusinessWeek

  12. Data Visualization: Modern Approaches – Smashing Magazine

  13. The 21 Heroes of Data Visualization: – BusinessWeek

  14. 20+ CSS Data Visualization Techniques – tripwire magazine

  15. MEDIA ARTS MONDAYS:Data Visualization Tools – PSFK

  16. 37 Data-ish Blogs You Should Know About – FlowingData

  17. 5 Best Data Visualization Projects of the Year – FlowingData

  18. 30 new outstanding examples of data visualization –

  19. Infosthetics: the beauty of data visualization – PingMag

  20. 5 Beautiful Social Media Videos – Mashable

Here are the top product type links in the field according to Twitter data between March 24 and Dec 31, 2009.

Top Data Visualization Product Links Mentioned on Twitter

  1. Axiis : Data Visualization Framework

  2. The JavaScript InfoVis Toolkit

  3. Microsoft – What is Pivot?

  4. Many Eyes

  5. Roambi – Your Data, iPhone-Style

  6. Flare – Data Visualization for the Web

  7. – For a fact based world view.

  8. SpatialKey – Location Intelligence for Decision Makers

  9. Tableau Software – Data Visualization and Business Intelligence

  10. SIMILE Widgets

and finally:

Top Data Visualization Websites Mentioned on Twitter

  1. Information Is Beautiful | Ideas, issues, concepts, subjects – visualized!

  2. FlowingData | Data Visualization and Statistics

  3. Information Aesthetics | Information Visualization & Visual Communication

  4. | A visual exploration on mapping complex networks

  5. DataViz on Tumblr

Charting the Beatles

O'Reilly Radar has a fine round-up of election mapping.
Nov 5th, 2008 by Tom Johnson

Mapping The Scenarios: Geo Resources For The Election

by Brady Forrest

nyt proportinal states map

If you're like me you'll be looking at maps and polls and news sites all day long on November 4th. As the polls close and some states go red vs. blue you'll be updating your mental map of the country. Below are some tools and data sources for simulating scenarios online.

New York Times – The Grey Lady has invested in a lot of great visualization tools. The map above shows the electoral importance of each state and their leanings. Their maps also let you create your own scenarios. So if you believe that Missouri will go Red change it and see how that effects the totals. If you want to reminisce check out their debate visualizations.

GeoCommons – An online geo-database and mapping tool has many pertinent data sets available including Early Voting Data and Active Registered Virginia Voters.

FiveThirtyEight – Nate Silver and his fellow bloggers have been doing a great job of reading the polls. They've called it for Obama, but this is based on their math not their affiliation. As states are tallied I am sure they'll be updating their simulations.

Electoral-Vote – Very similar to FiveThirtyEight, this site also aggregates polls, but applies a different algorithm. They even have a Data Galore section with CSV's of all data used on the site.

Twitter Vote Report – As mentioned yesterday this site will be collecting vote reports from people on the ground.

Google – If you want to see recent voting patterns, Google has made the US election since 1980 available. They've also released a number of other maps and an election portal. And of course don't forget Google Hot Trends.

Video Galore – If you want to watch the election news online Silicon Alley Insider and LifeHacker have collected the available streams.

2008 presidential election cartogram
Nov 5th, 2008 by Tom Johnson

 Mark Newman's 2008 presidential election cartogram page is available.

"Atlas of Cyberspace" book now free and online
Oct 22nd, 2008 by Tom Johnson

Martin Dodge writes….
A bit of a shameless plug but to let people know that the full content of
the Atlas of Cyberspace, a book I co-wrote in 2001, is now available for
free. You can download it as a pdf from,
While some of the content is out of date, I hope that it will provide an
interesting and informative historical record of how the Internet was
mapped and visualised in the mid to late 1990s.
Feedback and comments are most welcome.

Views from a NYTimes R&D guy
Oct 8th, 2008 by Tom Johnson

We suspect that The New York Times is the only newspaper in North America that has a research-and-development department. Clearly, that's a concept to far-out for America's newspaper industry leaders. How, Nick Bilton, who knows that department, writes in O'Reilly's Radar today:


Posted: 07 Oct 2008 09:05 PM CDT

Guest blogger Nick Bilton is with the New York Times R&D Lab during the day and NYC Resistor at night.


Working in the R&D Labs at The New York Times, I'm constantly asked, “How long will paper be around?” or more to the point, “When will paper really die?” It's a valid concern, and a question no one can answer with a timetable. But there will be a point–and I believe in our lifetime–when we'll see the demise of the traditional print newspaper. After all, paper is just a device. It provides a way to communicate information, just as a TV, radio, cell phone, and billboard do. This isn't to say that newspapers will go away. The way they are delivered will just change, and in turn, the narrative as we know it will have to adapt–more on this in a later post. But paper can easily be replaced–and the factor that will drive this is simple economics.

Let's put books and magazines aside for a moment, and focus on newsprint. The cost of printing a national newspaper like the Wall Street Journal is close to $150k a day. That's just for the newsprint. When you factor in printing plant rental or ownership fees, machine maintenance, shipping, and wages for plant employees, drivers, and packers, the final cost is hundreds of millions of dollars a year. Now if you have an average of 1,000,000 subscribers to the newspaper on a daily basis (this is a rounded-down average of a few top papers) and you stopped printing the paper, but instead gave your readers an eReader at $200 apiece, it would take fewer than six months for you to recoup your costs. If you factor back in books and magazines, people who read more than one newspaper a day, and throw in the odd journal or two, you've got a multi-billion dollar industry that could collectively save billions of dollars a year by moving away from ink on paper.

But there are problems associated with this model. There's the environmental effect–devices may not be as benign as they seem, after the impact of manufacturing, materials, and shipping is considered. There's a human cost–people who print and deliver the paper would lose their jobs. There are the immense difficulties of advertising on small, different-sized devices–do advertisers create one ad at one size, or many different ones, do they animate, etc. And then there's the issue that you have to treat the device with care, something you don't need to do with paper.


But for every argument against digital paper, eInk or whatever you want to call it, there is a rebuttal, or at least there will be over time. The simple fact that an eInk device today can carry a thousand books and that it only needs recharging once a month speaks paramount. The ability to download content over the air instantly–something that the “digital native” generation fully expects–is compelling. And as far as cost goes, this will be a non-issue in the coming years. Look at the cost of a 15 Megabyte hard drive 20-plus years ago, it was $2495! Today, you couldn't buy or find that size hard drive anywhere, and if you could it would cost mere pennies to create. I'm willing to bet that the cost of an eInk device will be negligible in 20 years.

A common response to the prospect of an eReader is, “But I love the feel of paper, I love a good book in my hands.” I can empathize with that sentiment, but I don't think the digital generation can. If it's not a touch screen, or hyperlinked, or instantly available at the press of a button, then it's not worth their time. And as soon as a reasonable iPod-like replacement comes along, paper won't be worth the publishing industry's time either.



Three Tuesdays workshop on data and the political campaigns at the Santa Fe Complex
Sep 27th, 2008 by Tom Johnson

Handicapping the Horserace

Published by Don Begley at 10:09 pm under Complex News, event

Handicapping the Horserace
    •September 30, 2008 – 6:30-8 pm  •October 7, 2008 – 6:30-8 pm  •October 14, 2008 – 6:30-8 pm

It’s human nature: Elections and disinformation go hand-in-hand. We idealize the competition of ideas and the process of debate while we listen to the whisper campaigns telling us of the skeletons in the other candidate’s closet. Or, we can learn from serious journalism to tap into the growing number of digital tools at hand and see what is really going on in this fall’s campaigns. Join journalist Tom Johnson for a three-part workshop at Santa Fe Complex to learn how you can be your own investigative reporter and get ready for that special Tuesday in November.

Over the course of three Tuesdays, beginning September 30, Johnson will show workshop participants how to do the online research needed to understand what’s happening in the fall political campaign. There will be homework assignments and participants will contribute to the Three Tuesdays wiki so their discoveries will be available to the general public.

Everyone is welcome but space will be limited. A suggested donation of $45 covers all three events or $20 will help produce each session. Click here to sign up.

  • The Daily Tip Sheet (September 30, 6:30 pm)

    Newspapers are a ‘morning line’ tip sheet. There isn’t enough room for what you need to know.

    Newspapers can be a good jumping-off point for political knowledge, but they rarely have enough staff, staff time and space to really drill down into a topic. Ergo, it is increasingly up to citizens to do the research to preserve democracy and help inform voters. Tonight we will be introduced to some of the city, state and national web sites to help in our reporting and to a few digital tools to help you save and retrieve what you find.
  • Swimming Against the Flow (October 7, 6:30 pm):

    How to track data to their upstream sources.

    A web page and its data are not static events. (Well, usually they are not.) Web pages and digital data all carry “signs” of where they came from, who owns the site(s) and sometimes who links to the sites. We will discuss how investigators can use these attributes to our advantage, and also take a step back to consider the “architecture of sophisticated web searching.”
  • The Payoff (October 14, 6:30 pm)

    Yup, it IS about following the money. But then what?

    Every election season, new web sites come along that make it easier to follow the money — election money. This final workshop looks at some of those sites and focuses on how to get their data into a spreadsheet. Then what? A short intro to slicing-and-dicing the numbers. (Even if you are a spreadsheet maven, please come and act as a coach.)

This workshop is NOT a sit-and-take-it-in event. We’re looking for folks who want to do some beginning hands-on (”On-line hands-on”, that is) investigation of New Mexico politics. And that means homework assignments and contributing to our Three Tuesdays wiki. Participants are also encouraged to bring a laptop if you can. Click here to sign up.

Tom Johnson’s 30-year career path in journalism is one that regularly moved from the classroom to the newsroom and back. He worked for TIME magazine in El Salvador in the mid-80s, was the founding editor of MacWEEK, and a deputy editor of the St. Louis Post-Dispatch. His areas of interest are analytic journalism, dynamic simulation models of publishing systems, complexity theory, the application of Geographic Information Systems in journalism and the impact of the digital revolution on journalism and journalism education. He is the founder and co-director of the Institute for Analytic Journalism and a member of the Advisory Board of Santa Fe Complex.


Call for papers on urban data visualization
Sep 24th, 2008 by Tom Johnson    

VISUALIZAR'08: Database City – Call for Papers    

Open call for the presentation of theoretical works on data visualization focused on the city context. A maximum of five papers will be selected to be presented during  VISUALIZAR'08: Database City Seminr (November 3 and 4, 2008).

Deadline: October 5, 2008.   


VISUALIZAR'08: Database City Seminar  

Curated by: José Luis de Vicente Dates: November 3-4, 2008

Venue: Medialab-Prado, Madrid (C/ Alameda, 15 ∙ 28014 Madrid, Spain)        

INTRODUCTION     Data Visualization is a transversal discipline which harnesses the immense power of visual communication in order to explain, in an understandable manner, the relationships of meaning, cause and dependency which can be found among the great abstract masses of information generated by scientific and social processes.   Visualizar, one of Medialab-Prado's lines of work, is directed by José Luis de Vicente, and is conceived as an open and participartory research project around theory, tools and strategies of information visualization.  

VISUALIZAR'07 was held for the first time in November 2007 and explored the social, cultural and political possibilities of the art and science of data visualization. This year, VISUALIZAR'08: Database City will have the city as its sole focus.   Urban environments, which are becoming increasingly dense, complex and diverse, are one of contemporary society’s largest “databases”, daily generating volumes of information that require new methods of analysis and understanding.  

How can we use the data visualization and information design resources to understand the processes governing contemporary cities and better manage them? What can we learn from studying traffic and pedestrian movement flows through the streets of Madrid? What would happen if we filled the streets with screens providing information updated each moment about water and electricity consumption?  

For two weeks, lectures, presentations, and an intense project development programme will involve participants from all over the world in a collaborative process that will culminate in eight new proposals for the city.



»  Substance:WordPress   »  Style:Ahren Ahimsa