The Internet, Data and Phil Meyer
Apr 3rd, 2008 by Tom Johnson

Last week we had the opportunity to participate in a symposium honoring Phil Meyer, Knight Chair of Journalism at the University of North Carolina-Chapel. About 30 journalism educators, practitioners and former students of Phil's spent the better part of two days kicking hard on the topic “”Raising the Ante: The Internet's Impact on Journalism Education and Existing Theories of Mass Communication.”

Kathleen Frankovic, director of surveys for CBS News, used the symposium as the basis for her column last week at The column (not quite a blog) is called “Poll Positions” (about public opinion, polls and the process of conducting and reporting them). Here is her column's link:

Kathy's column makes reference to something we wrote on the topic of the future of polling. That short paper, but with all of its hyperlinks, can be found at “Are We Researching How To Do Research?”

On Thursday night Phil gave a short speech at a dinner attended by the participants and his family. In “Something strange and possibly dangerous,” he highlighted that change is not coming, it is here, and that all of us have to change out thinking and practices if democracy is to survive.

“We need to turn our conversation toward an economic theory of journalism. We need to apply existing theory to understanding the processes and effects of the new media. We need to learn how to sell enlightened understanding to the public so that it can preserve its democratic values. The synergy of mass media and mass production is gone, probably forever. Something strange – and possibly dangerous — is taking its place.”

Strange and dangerous and something most exciting.

The Dataweb and the DataFeret
Jan 3rd, 2008 by Tom Johnson

Marylaine Block's always informative “Neat New Stuff” [Neat New Stuff I Found This Week at] tipped us to the DataWeb site and its interesting tool, the Data Feret (or “dataferet”).

“TheDataWeb is a network of online data libraries that the DataFerrett application accesses the data through. Data topics include, census data, economic data, health data, income and unemployment data, population data, labor data, cancer data, crime and transportation data, family dynamics, vital statistics data, . . . As a user, you have an easy access to all these kinds of data. As a participant in TheDataWeb, you can publish your data to TheDataWeb and, in turn, benefit as a provider to the consumer of data.”

What is the DataFerrett?
DataFerrett is a unique data mining and extraction tool. DataFerrett allows you to select a databasket full of variables and then recode those variables as you need. You can then develop and customize tables. Selecting your results in your table you can create a chart or graph for a visual presentation into an html page. Save your data in the databasket and save your table for continued reuse. DataFerrett helps you locate and retrieve the data you need across the Internet to your desktop or system, regardless of where the data resides. DataFerrett:
* lets you receive data in the form in which you need it (whether it be extracted to an ascii, SAS, SPSS, Excel/Access file); or
* lets you move seamlessly between query, analysis, and visualization of data in one package;
* lets data providers share their data easier, and manage their own online data.
DataFerrett Desktop IconDataFerrett runs from the application icon installed on your desktop.

Check it out at


JAGIS at The University of Hong Kong
Dec 16th, 2007 by Tom Johnson

What have we here? Cooperation between two academic departments in the same university? Largely unheard of in most schools, but it has happened with positive results in Hong Kong.

23 Nov 2007

Power Distribution of the Four Political Camps, Seeing the 2007 District Council Election Results with Maps

The Department of Geography and the Journalism and Media Studies Centre of The University of Hong Kong (HKU) announced today (November 23) an analysis of results of the 2007 District Council Election of four political camps from the spatial perspective.

Dr. P.C. Lai, Associate Professor of the Department of Geography, and her team applied the Geographic Information System (GIS) to analyze results of the District Council Election. The GIS technology was used to explore the power re-distribution of the four political camps or affiliations – pro-government, pro-democrat, moderate (Liberal Party) and independent candidates – of the said election. [more]

If you're really serious about searching….
Dec 5th, 2007 by Tom Johnson

Deep Web Research 2008

Bots, Blogs and News Aggregators is a keynote presentation that I have been delivering over the last several years, and much of my information comes from the extensive research that I have completed over the years into the “invisible” or what I like to call the “deep” web. The Deep Web covers somewhere in the vicinity of 900 billion pages of information located through the world wide web in various files and formats that the current search engines on the Internet either cannot find or have difficulty accessing. Search engines currently locate approximately 20 billion pages.

In the last several years, some of the more comprehensive search engines have written algorithms to search the deeper portions of the world wide web by attempting to find files such as .pdf, .doc, .xls, ppt, .ps. and others. These files are predominately used by businesses to communicate their information within their organization or to disseminate information to the external world from their organization. Searching for this information using deeper search techniques and the latest algorithms allows researchers to obtain a vast amount of corporate information that was previously unavailable or inaccessible. Research has also shown that even deeper information can be obtained from these files by searching and accessing the “properties” information on these files.

This article and guide is designed to give you the resources you need to better understand the history of the deep web research, as well as various classified resources that allow you to search through the currently available web to find those key sources of information nuggets only found by understanding how to search the “deep web”.

This Deep Web Research 2008 article is divided into the following sections:



More on the SoCal fire coverage
Oct 25th, 2007 by Tom Johnson

This comes from the Poynter blog…..

Posted by Amy Gahran 5:42:13 PM
CA Wildfire Coverage: Intriguing Online Approaches

KPBS San Diego is offering fire news updates via Twitter — possibly the best use of this service I've ever seen.

While much of Southern California burns, online news staffs and citizen journalists definitely aren't fiddling around. Here's a quick roundup of some of the more intriguing efforts:

What kinds of innovative online coverage of the fires are you seeing today? Please comment below.

(Thanks to the members of Poynter's Online News discussion group for tips to some of the items above.)


Zotero: I think they've got it this time
Oct 5th, 2007 by Tom Johnson

Yes, call us fickle and lacking in loyalty when it comes to note-taking and research organization tools.  Does anyone else remember the 5×8 cards with holes punched on all four perimeters?  You entered “tags” or keywords by clipping out the outer edge of the hole, and when you needed to find a particular note card, a knitting needle-sized wire was inserted into the whole pack.  Shake the cards and the desired note fell out.  Sometimes.

Since going digital 25 years ago, we've tried dozens of tools to try and bring some order to what we've turned up online and need to save.  Most were fine innovations and advances at the time, but there was often something that didn't quite meet all of our needs or desires.  That still might be true, but a new entry in the research management derby (thanks to the cite from The Scout Report quoted below) delivers up an impressive new tool.

Zotero is a Firefox extension with rich, intuitive tools that are flexible enough to support the way YOU want/need to work.  This is only version 1.0, but I think I have a new best friend.


“It can be hard to keep Tom Wolfe and Thomas Wolfe straight at times, and if you are working on an academic paper that incorporates both of these august characters, you probably want to keep those research sources in good order. Thanks to Zotero, it is very easy to do just that. Zotero is a Firefox extension that helps users collect, manage, and cite their research sources. Zotero can automatically capture citation information from web pages, store PDF files, and also export these citations with relatively ease. This very helpful extension is compatible with computers running Firefox 2.0.” [KMG]


The "Traditional Future" of library research
Sep 18th, 2007 by Tom Johnson

From O'Reilly Radar's Publishing blog comes this interesting item. See

The Traditional Future

“A prominent U.S. sociologist and student of professions, Andrew Abbott of the University of Chicago, has written a thought-provoking thesis on what he terms “library research” — that is, research as performed with library-held resources by historians, et. al, via the reading and browsing of texts — compared to social science research, which has a more linear, “Idea->Question->Data->Method->Result” type of methodology.

“The pre-print, “The Traditional Future: A Computational Theory of Library Research,” is full of insights about library centric research, including intriguing parallels between library research and neural net computing architectures; a comparison that made me think anew, and with more clarity, about how the science of history is conducted. Armed with a distinctive interpretation of library research, Abbott is able to draw some incisive conclusions about the ramifications of large repositories of digitized texts (such as Google Book Search) on the conduct of scholarship…”


More on Benford's Law
Jul 30th, 2007 by JTJ

We've long been intrigued with Benford's Law and its potential for Analytic Journalism.  Today we ran across a new post by Charley Kyd that explains both the Law and presents some clear formulas for its application.

An Excel 97-2003 Tutorial:

Use Benford's Law with Excel

To Improve Business Planning

Benford's Law addresses an amazing characteristic of data. Not only does his formula help to identify fraud, it could help you to improve your budgets and forecasts.

by Charley Kyd

July, 2007

(Email Comments)

(Follow this link for the Excel 2007 version.)

Unless you're a public accountant, you probably haven't experimented with Benford's Law.

Auditors sometimes use this fascinating statistical insight to uncover fraudulent accounting data. But it might reveal a useful strategy for investing in the stock market. And it might help you to improve the accuracy of your budgets and forecasts.

This article will explain Benford's Law, show you how to calculate it with Excel, and suggest ways that you could put it to good use.

From a hands-on-Excel point of view, the article describes new uses for the SUMPRODUCT function and discusses the use of local and global range names.  [Read more…]



The Beauty of Statistics
Jul 11th, 2007 by JTJ

FYI: From the O'Reilly Radar

Unveiling the Beauty of Statistics

Posted: 11 Jul 2007 03:01 AM CDT

By Jesse Robbins

I presented last week at the OECD World Forum in Istanbul along with Professor Hans Rosling, Mike Arrington, John Gage and teams from MappingWorlds, Swivel (disclosure: I am an adviser to Swivel) and Many Eyes. We were the “Web2.0 Delegation” and it was an incredible experience.

The Istanbul Declaration signed at the conference calls for governments to make their statistical data freely available online as a “public good.” The declaration also calls for new measures of happiness and well-being, going beyond just economic output and GDP. This requires the creation of new tools, which the OECD envisions will be “wiki for progress.” Expect to hear more about these initiatives soon.

This data combined with new tools like Swivel and MappingWorlds is powerful. Previously this information was hard to acquire and the tools to analyze it were expensive and hard to use, which limited it's usefulness. Now, regular people can access, visualize and discuss this data. Creating an environment where knowledge can be shared and explored.

H.G. Wells predicted that “Statistical thinking will one day be as necessary for efficient citizenship as the ability to read or write.” Proponents of specific public policies often use statistics to support their view. They have the ability to select the data to fit with the policy. Democratization of statistics allows citizens to see the data that doesn't fit the policy, giving the public the power to challenge policymakers with new interpretations.

I highly recommend you watch Professor Rosling's exceptional summary of these exciting changes (where I got the title for this post), as well as his talks at TED.”



Impact of feedback in mass media message.
Jun 30th, 2007 by JTJ

A recent article worth a look over by the journalism community. What we do DOES have impact.

Juan Carlos González-Avella, Mario G. Cosenza, Konstantin Klemm, Víctor M. Eguíluz and Maxi San Miguel (2007)

Information Feedback and Mass Media Effects in Cultural Dynamics

Journal of Artificial Societies and Social Simulation vol. 10, no. 3 9

PDF at
Received: 11-Jan-2007 Accepted: 18-May-2007 Published: 30-Jun-2007

We study the effects of different forms of information feedback associated with mass media on an agent-agent based model of the dynamics of cultural dissemination. In addition to some processes previously considered, we also examine a model of local mass media influence in cultural dynamics. Two mechanisms of information feedback are investigated: (i) direct mass media influence, where local or global mass media act as an additional element in the network of interactions of each agent, and (ii) indirect mass media influence, where global media acts as a filter of the influence of the existing network of interactions of each agent. Our results generalize previous findings showing that cultural diversity builds up by increasing the strength of the mass media influence. We find that this occurs independently of the mechanisms of action (direct or indirect) of the mass media message. However, through an analysis of the full range of parameters measuring cultural diversity, we establish that the enhancement of cultural diversity produced by interaction with mass media only occurs for strong enough mass media messages. In comparison with previous studies a main different result is that weak mass media messages, in combination with agent-agent interaction, are efficient in producing cultural homogeneity. Moreover, the homogenizing effect of weak mass media messages is more efficient for direct local mass media messages than for global mass media messages or indirect global mass media influences. Keywords: Agent Based Model, Culture, Dissemination, Mass Media

»  Substance:WordPress   »  Style:Ahren Ahimsa