Guest post by Amanda Wilson of UPI Next
There are some basic journalism standards – accuracy, balance, good storytelling – that are sacrosanct to journalists across the world. That is the idea behind diverse attempts to create tools for evaluating journalism content produced in newsrooms or in media development projects.
In the media development arena, many project funders require specific measurements of content’s improvement and impact. The evaluation process often provides results that are qualitative, and not comparable, meaning that any true insight into “impact” is lost. Is it possible to use numbers to measure the quality of journalism stories?
United Press International (UPI) has set out to build a tool that could do just that.
Since 2007, UPI president Nicholas Chiaia has leveraged company resources and infrastructure to educate and empower the next generation of media professionals. More recently, he has focused the company’s media development efforts on practical journalism education approaches, exploring technologically driven tools for media development.
“I am passionate about a sustainable, contemporary, and culturally contextualized approach that empowers news gatherers of all stripes,” Chiaia said.
In 2012, UPI launched a project to interview veteran journalists, editors, and media trainers. The goal was to build a metric that could collect data for funders, show “change” in quality of a story using quantitative scores, and get results back to journalists. The resulting resource, the UPI Content Evaluation Tool ™, aims to distill the mental metrics that editors use and standardize them in a news story evaluation tool.
“We have exciting plans for scaling our Evaluation Tool to reach a much broader group of aspiring journalists worldwide,” Chiaia said.
The tool works much like a score sheet, with subcategories of quality journalism broken down and described in great detail to eliminate ambiguity in scoring. The UPI Content Evaluation Tool™, also includes space for targeted comments under each category that can be delivered to journalists.
A consultant for UPI’s media development division, Lisa Schnellinger of Near Media LLC, a journalist and media trainer who has worked in 21 countries, has been a vocal advocate for an “evaluation feedback loop” that could benefit funders’ needs for specific data and journalists’ needs to be brought onboard as stakeholders too. She said many traditional journalists resist the idea that good journalism can be quantified.
“They think of it as an art and they believe that you can’t measure an art,” Schnellinger said. “I understand that, I really do, but I simply think that it is not true in the case of journalism.” She said a good tool simply replicates what already happens in a good editor’s head. “That is the key for journalists to understand: this is not some external academic exercise, it actually derives from the thought process of an editor.”
The idea is one that Maureen Taylor, professor of communications at the University of Oklahoma, has also advocated. She lent her expertise to the development of UPI’s tool.
UPI used the Content Evaluation Tool during implementation of a project focusing on elections and political accountability reporting in Pakistan. Working journalists in Pakistan were paired with an international, professional mentor who guided them through the reporting and writing process for a story. At each step of their reporting, the journalists’ stories were evaluated on UPI’s content mentoring platform using the UPI Evaluation Tool. When mentors were satisfied the stories merited publication, the journalists’ articles could be submitted toUPI.com for international publication. In total, 98 stories by Pakistani journalists were mentored to publication on UPI.com.
One of the reporters, a female journalist from Lahore, went on to be selected for a 2014 World Press Fellowship. A Pakistani evaluator commented after using the Tool that it helped him by giving him a checklist of the criteria for a comprehensive and well-rounded story.
“The progress that I have noted among reporters from one draft of the story to the next has been remarkable,” he said. More importantly, the tool helped him accomplish the dream of many editors: “What has been most thrilling is that more and more often I’ve received stories for evaluation which were better-written even in the first draft.”
Comments (0)