Assessing Impact of Media

Back to Search

Analysis

Media Impact Focus at Paley Center Reveals Wealth of Assessment Approaches

It’s clear that media productions and publications can drive social impact — but how? That was the question posed to speakers and participants at the November 13 Media Impact Focus event: Assessing the Impact of Media

As it turns out, there’s no single answer, but many. Throughout this insight-packed day at The Paley Center for Media, leading analysts, producers, and national network leaders presented an array of leading-edge tools and frameworks. We’ll explore several  of these tools in more depth throughout this Assessing the Impact of Media (AIM) section in the coming weeks.

Growth in Foundation Support for Media in the United StatesThe blossoming of new approaches reflects the expansion and range of the media funding field, as revealed by Growth in Foundation Support for Media in the United States, produced by the Foundation Center in collaboration with Media Impact Funders and the John S. and James L. Knight Foundation. Released in conjunction with an unprecedented data visualization tool for researching media funding, and an Issue Lab featuring the latest reports on the field—both hosted here on our new web site—the report reveals that between 2009-2011, foundations invested $1.86 billion in media projects.

As the Foundation Center’s Seema Shah told attendees, the report serves as baseline to track the rising tide of support for media: 1,012 foundations made 12,040 media-related grants across five areas: journalism, news, and information; media access and policy, media applications and tools, media platforms, and telecommunications infrastructure. Media-related grant-making is expanding at nearly four times the rate of domestic giving for other areas, including science and technology religion, and the social sciences.

“Overall I think the message is that the field is large and growing,” said Media Impact Funders Executive Director Vincent Stehle. “And that is reason for optimism.”

“That only underscores the importance of our meeting here today, which is intended to identify and explore lots of different approaches to assess and demonstrate the impact that media is having on society. The field of impact assessment is itself large and growing.”

What’s True, and What’s New?

The day’s discussions confirmed an emerging consensus on several points in the impact debate:

  • Data is now rampant—but not everything that can be counted counts, and vice versa. Both quantitative and qualitative approaches are needed to understand and tell the story of a media project’s impact.
  • It’s crucial to track outcomes, not just outputs—what’s more, “engagement” is not always tantamount to impact.
  • Audience size matters, but is not the only factor in evaluation. Projects designed to innovate new approaches, reach diverse audiences, increase digital literacy or reveal hard truths might not be best judged primarily on reach.
  • Media impact analysis should not address simply effects on individual consumers, but on frames, systems, institutions, and in some cases, the field of media production itself.
  • There are important lessons to be drawn from adjacent fields—including design thinking and agile development—about how to tie metrics to project strategy and development.

At the same time, there are still many open questions and points of contention, which we will be exploring in this AIM section. To get the ball rolling, here are a few “newisms” we’d offer to bolster the emerging “truisms” listed above:

  • Public interest media can make an impact—the question is not “If?” but “So what?”
  • The process of developing collective understanding about impact among funders and grantees should not operate like a funnel, leading to a single method or equation. Instead it should be a dynamic networked process, with various hubs testing and refining different methods tailored to specific processes.
  • Open and transparent sharing of models, impacts, successes and failures will help to clarify and build this field.
  • Impact evaluation should be practical, affordable, relevant, and actionable.

Interested in keeping pace with this growing field? We’ll be sending out regular email updates on resources and tools in coming weeks. Email us to subscribe.

Working Across Silos

What's your so what?

The event’s plenaries and workshops focused on two tracks: film and entertainment, and journalism. The day was structured to both highlight distinctions between how practitioners and funders working in each field are evaluating impact, and to allow for cross-pollination of approaches. A “collaboratory” at the day’s end provided a chance for attendees to connect in order to put the day’s ideas into practice, and a pair of “Wildcard” sessions offered the chance for small-group discussions on the Foundation Center report and transmedia activism.

Host Pat Mitchell, President and CEO of The Paley Center for Media introduced two leading media impact experts to kick off the day’s tracks: Chad Boettcher, the Executive Vice President of Social Action & Advocacy at Participant Media, and Charles Lewis, Executive Editor of the Investigative Reporting Workshop

Mitchell noted that The Paley Center’s history is all about examining how media “moves the needle” on important issues, and showed a clip from the Center’s extensive archive featuring CBS’s influential report on Silent Spring. Since that report brought the environmental issue to mainstream America, Mitchell observed, journalists have “continued to beat the drum, raise the awareness, and put the issues in front of the public.”

Chad BoettcherBoettcher unveiled breaking research from The Participant Index (TPI), a comparative survey panel for analyzing the impact of both feature and documentary films. With support from the Gates Foundation, Participant Media has been working with the Norman Lear Center to survey a 50,000-person representative consumer panel about how films are influencing behavior change, social actions, and policies. One of the goals of this research, he says, is to discover the “word of mouth moments” that help drive a film into the national discourse.

In a benchmark study, the TPI team polled respondents on 71 media properties, ranging from documentaries, to features, TV narratives, reality shows, and online videos.  A third of respondents reported watching “pro-social” entertainment—but even responses to feature films such as Magic Mike indicated that the film spurred some small measure of social awareness and action. Of course, many respondents did not take any actions. “Not everyone is doing something, Boettcher said, “But we are interested in getting to know those people who are.”

Film & Entertainment workshop

 

Boettcher’s presentation kicked of a series of examinations throughout the day of other tools and frameworks that makers and funders are using to track media impact. Check back on this blog for additional details on TPI, as well as the Harmony Institute’s ImpactSpace, the ConText tool being developed by Assistant Professor Jana Deisner’s team at University of Illinois with support from the Ford Foundation, and the Impact Reports developed by BRITDOC. On our site, you’ll also find reports, articles and high-impact media projects produced by the day’s other film and entertainment speakers.

Chuck Lewis

To round out the morning keynotes, Mitchell interviewed Lewis, whose latest report is titled Measuring Impact: The Art, Science and Mystery of Nonprofit News. Journalism is undergoing a sea change, Lewis said. In the 20th century, “You could do powerful, in-your-face journalism, and you could change things, but you didn’t have to set out to change it, and you didn’t track it while you did it.”

Now, with the shrinking of commercial newsrooms and the rise of nonprofit outlets, dynamics have shifted. Lewis described the opportunity to measure journalism’s impact as “an opportunity to increase value….It’s very complex, but I do think it is doable.” He noted that privacy concerns arise hand-in-hand with the new culture of data-driven reporting.

Lewis also suggested that despite calls for a single impact method or standard, there is no one-size-fits-all formula.“You try to do what you’re supposed to be doing, you embrace the new technology, and sometimes you get really lucky and it’s thrilling, he said. “But sometimes you do the best you can and no one notices”

His remarks kicked off a lively dialogue in the morning journalism workshop, where attendees puzzled through a variety of definitions and frameworks for defining impact. In contrast to the film and entertainment discussions—which, after years of field-building, now focus more directly on how media can move audiences and policymakers to action— leading journalism thinkers are still debating the role that reporting should play in engaging audiences.

Catch up on the competing perspectives, including a piece by morning workshop presenter Jon Sotsky of the Knight Foundation, in our AIM articles section.

Lunchtime speaker David Corn picked up the question of journalism’s power to move public opinion and ran with it, unpacking the effect of the release of the 47% video by Mother Jones on 2012 election reporting. Check our Insights section in the coming days for the full text and video from his talk.



The afternoon workshop examined how national journalism networks—including the Media Consortium, the Investigative News Network, and the Solutions Journalism Network—are collaborating to prototype tools that demonstrate impact. Learn more about INN’s impaq.me tool, and watch our analysis section for further details on the research that The Media Consortium has been conducting with Gary King of Crimson Hexagon.

Want to learn more about the day’s dialogue? Watch the livestream from the plenaries, or send your questions to Jessica Clark.