One
of the insights to the craft that business reporters learn early in the
game is that the key to understanding annual reports is to read the
footnotes and endnotes. That's where the juicy stuff is. So
it is, it seems, for educational reporters.
A story in Sunday's St. Paul (Minn.) Pioneer-Press by higher education writer Paul Tosto, “'Home alone' data debatable” points out the importance of reading the footnotes.
Backstory: In June, a group called the Minnesota Commission on Out-of-School Time released a report claiming “Minnesota has the nation's highest
percentage of teens home alone each afternoon. It has more young
children taking care of themselves after school than any state in the
country. Half its kids aren't part of any structured after-school
activity.”
Tosto read the report, scratched his head and then looked at the
footnotes. Ultimately, he found the data and sourcing for the
Commission's report didn't hold up. Here's what Tosto had to say
about how he picked up the scent of the story:
“My concerns about the Minnesota Commission on Out-of-School Time
findings surfaced when the report came out June 2. The sweeping nature of one statement, “Minnesota is
home to 950,000 young people and has the highest percentage in the country of
children ages 12 and older alone at home every single afternoon” startled me.
That was going to lead my story.
But when I tried to
trace back the footnote, I
found the Web link that was supposed to provide
the source for the information didn't work. When
I asked for clarity, I was sent information about 10- to 12-year olds, not
teenagers, and the data was from 1997 and involved only 13 states.
I became worried enough about it that day that I didn't write anything
on the report or its release.
I spent the next few weeks on and off asking the
commission's chief of staff for more information, trying
to nail down three key pieces of information the
group was using.
With the first finding, they eventually acknowledged
to me that they did not have data showing Minnesota as the state with
“the highest percentage in the country of
children ages 12 and older alone at home every single afternoon.” Somone had apparently confused information from a couple of
reports.
With the second finding — Minnesota has the
country's highest percentage of 10- to 12-year-olds caring for themselves after
school — I went back to the origins of that data, calculations by the Urban
Institute of data from the 1997 Survey of America's Families.
Minnesota did have the highest percentage of children
reported in self care and it was much higher than the national average the Urban
Institute had calculated. But when I talked to one Urban Institute researcher
who'd worked with the data, she said it was incorrect to say that Minnesota had
the highest in the country since the data involved only 13 states. And surveys
done by Minnesota's Wilder Foundation just a couple of years later showed
percentages of children in self care that were much smaller than the Urban
Institute report.
With the third finding — “about half” the state's children were not part of a structured after
school activity — I had concerns about the
methodology.
The commission's press release initially cited a
report by one of its researchers a year earlier as the source. When I looked at
that report, I found essentially unscientific discussion groups conducted by the
researcher at nine sites across the state. Only 101 kids participated and the
demographics did not reflect Minnesota's race and ethnicity. When I raised
questions about it, the commission said (despite its press release) that it
didn't base its conclusion on those site visits. But the commission did not
provide any local, scientific data to back it up.”
Very nice work by a reporter who simply asked: “What do we [in this case, they,] know and how do we know it.
