“Good enough” isn’t good enough if you are a data journalist. Even “accurate” isn’t good enough. When you’re working with data, it’s important to run it through a bias filter at every stage.
That’s what three highly regarded investigative journalists insisted as we prepared for a panel on the ethics of working with data at the Investigative Reporters and Editors conference in June. They were joined by civil rights attorney Eva Jefferson Paterson, president of the Equal Justice Society in Oakland, Calif., in highlighting the dangers of skewed data that could undermine investigative work. Reporters are vulnerable at each step of gathering, interpreting and presenting data, they said.
Journalists commonly guard against data weighted by sources to make a point. But most assume that raw data itself is impartial. That’s not the case, Paterson said. Experimental psychologists are finding that implicit bias can creep into the collection and interpretation of what appear to be straightforward numbers. Our eyes can trick us, especially when we are working with data that trigger our unintentional attitudes about social groups.
“Objective information can be skewed because of our implicit bias about people of color,” Paterson said, urging journalists to consider the implications for their reporting. As an example, Paterson described studies of fingerprint matches by forensic scientists. In one, individual evaluators were asked to re-examine prints that they didn’t know they had matched before. When they were offered potentially biasing “case information,” most made completely different judgments. “Confirmation bias” causes us to choose according to our expectations.
Ricardo Sandoval -Palos, senior editor of NPR’s “Morning Edition,” suggested that journalists avoid thinking about data as proof. It should be a setup for further questions, he said. And those questions should not necessarily be the obvious ones. Think about the population the data represent and develop a familiarity with that group.
“Demographically, am I understanding the right questions to ask?” Sandoval Palos said. “Be skeptical.”
Sandoval -Palos pointed to data-based stories that invite misinterpretation. An Associated Press story about the high proportion of Latinos in federal detention was nuanced, offering vital information about immigration laws that drive the phenomenon. But other reports using the same data made it sound as if Hispanics were unusually prone to crime.
It comes down to making sure your data is fair and accurate, agreed Jennifer LaFleur, senior editor for data journalism at the Center for Investigative Reporting. First, the original data sets should be checked for missing data, duplicates and internal problems. Then use your reporting to check out what you think you found.
LaFleur sends her summary data to the person or organization she is reporting on, she said, and often ends up with a sharpened analysis and a better story. Invite feedback and corrections when you publish, too.
Inaccurate, misleading data can do a lot of damage, said Chrys Wu, developer advocate at The New York Times, who is both journalist and coder. She called attention to data presentation at publication. It’s easy to get carried away with the bells and whistles of presenting visual information, she said, forgetting the central purpose: effective communication. Clever graphics can create confusion.
“If we accept that our job is to convey information, to convey truth, we need to make sure the people who see it understand exactly what we’re trying to present,” she said.
With the advent of great new tools to present data, consider the audience. How do we present information that allows people to think about complex problems in a time- and attention-short environment? The long life of material on the Internet makes accuracy all the more important.
Do a gut check when you’re done, LaFleur concluded at the panel. A key question, she said, is “What else could explain (these) findings?”
For a good resource on using and visualizing data, and things you should consider before publication, check out visualizingdata.com.
Sally Lehrman is the Senior Fellow for Journalism Ethics at the Markkula Center. She is an independent journalist who reports on science and social issues.
Tagged under: diversity
