Assessing Impact of Media

Back to Search

Analysis

The Walton Family Foundation: A journey from nonmedia funder to nontraditional media funder

By Kristin Tracz | program officer, Walton Family Foundation
Christine Schneider | senior communications officer, Walton Family Foundation
Drew Jacobs |  strategy, learning and evaluation officer, Walton Family Foundation

Although we at the Walton Family Foundation (WFF) wouldn’t characterize ourselves as media funders, in early 2017 we came to realize that we were making significant investments in media across all three of our program areas—K-12 education, the environment, and the Foundation’s home region of Northwest Arkansas and the Arkansas-Mississippi delta.

While funding media writ large isn’t a goal of WFF, we do want to see more attention paid to our focus areas, whether it be about school choice or coastal restoration. Our team has come to view grantmaking to journalism organizations as a way to increase media coverage about the issues we care about in order to shine a spotlight and move the needle on these topics.

But as we invested more in media organizations, we wanted to deepen our evaluation approach. After all, as an organization, we value continuous improvement and learning. We tend not to make general support grants; instead, we work with grantees to set a course and develop measures for success over the course of a grant in order to learn what worked and where adjustments may be needed in the future. This internal examination led to the creation of a resource we’re now pleased to share with our philanthropic peers: our new Impact Toolkit and Primer.

Building on a culture of learning

With all of our investments, we ask ourselves questions to understand the focus of our work, including:

  • Who are we trying to reach?
  • How much progress do we expect to see?
  • How should we measure success in this grant and beyond?
  • What evidence do we need to know that the work of a particular grant is having an impact?

The questions about what we should focus on, why, and what matters kept coming up. We needed a framework—a way to think through the logic and subsequently plan out metrics that would be more meaningful for us and for our grantees.

From the program officer perspective, impact was often a “I know it when I see it” phenomenon. When content goes viral, elicits reactions from public officials or shows up in interest group newsletters, it’s easy to see success. But we found it difficult to set targets and understand the impact of more regular drum-beat stories that—while not getting splashy, national, front-page coverage—we believe to be valuable.

The first step in this project was to see if anyone else had figured out the media impact question. After connecting with Grantmakers for Effective Organizations, Media Impact Funders, and other networks, we came to appreciate that this is a question our peers in philanthropy are currently struggling with.

As an internal workgroup representing the Walton Family Foundation, we issued an RFP and reviewed existing literature, which brought us to working with Lindsay Green-Barber, founder and CEO of Impact Architects. She works with funders, media and social change organizations to develop impact frameworks, strategies for maximizing impact and research methods for evaluating success. To get our heads around the issue, we took inventory of our existing investments across K-12 education, environment, home region, and evaluation and made the decision to focus our inquiry on journalism/media specific investments as opposed to broader advocacy and communications.

Working with Impact Architects, we developed a framework that begins with prompting program officers to articulate what a potential investment seeks to do for their work through a series of guided questions. For example, is an investment’s goal to change a policymaker’s mind? To increase a target population’s understanding of an issue? Or perhaps it’s meant to flood the news cycle and change the conversation about an issue.

From there, we worked through subsequent steps that are necessary for a program officer to take to determine the most meaningful ways to measure success and inform learning over the grant term.

What we discovered

  • This is upfront work. Careful thinking and planning with grantees at the outset helps both parties outline what is meaningful. It allows the us—the funder—to understand the true capacity of the grantee organization and to identify key metrics to track in order to better understand the impact that we both aim to achieve.
  • Standardize terms. We created a glossary (Page 22 of the toolkit) to communicate clearly what WFF is looking for and ensure we’re collecting useful data.
  • Develop indicators. We created a bank of common indicators, with definitions, and sample targets (“how much”) to help orient grantmaking staff and to give them a common jumping-off point. (Page 19 of the toolkit.)
  • Think more broadly about the capacity of recipient organizations. Different organizations have different capacity, different software, different analytics platforms, and different organizational culture around impact. We recognize that WFF has a role to play in building organizational capacity, not only so that these organizations can provide us with information, but also so they can do their work, better.
  • Provide the resources needed to succeed. Amplify what’s there. If we want to reach more people on social networks, we need to include budget for paid promotions. If we want to get coverage in other media outlets, we need to support partnerships. Maximizing impact may require supporting more than the reporting.

Planning means we’re now better positioned to use more focused and more meaningful metrics to assess our investments in media. We also recognized that this learning is iterative and that we’ll need to refine and reapply our framework continuously, especially as the landscape continues to change, be it in the analytics space or in terms of platforms and outlets.

So, how’s it going so far? We can benchmark our work in both the K-12 education reform and the environment space. We have found that lifting the hood in collaboration with a strong grantee partner has been hugely beneficial in building trust. The toolkit provides us with a strong foundation and shared language to evaluate past performance and consider new projects with media grantees. This opens up even more creative possibilities for measuring impact.

One particular highlight is an investment we made with the Society of Environmental Journalists (SEJ) and the New Orleans Times-Picayune/Nola.com to enhance the paper’s coastal coverage through the creation of a dedicated coastal desk. Because of the shared agreement that communicating the story of coastal Louisiana’s land-loss crisis and the science-based solutions that can address the challenge to a broader audience regularly was important, the initiative’s success exceeded our most ambitious goals. Given the caliber and deep credibility of the reporters, the Times-Picayune/Nola.com coastal desk was able to develop a partnership with the New York Times, which culminated in a 12-page special section that ran concurrently in both papers. Additionally, the Times-Picayune is now developing programming, including in-person events and social media conversations, to augment the written reporting. And, because of the up-front clarity of goals and measurement strategies, we are able to track audience engagement across multiple platforms.

Moving forward

While this initial phase of developing a media investment and impact measurement framework has resulted in our Impact Toolkit and Primer, we are already asking ourselves how we can take another step forward. We think there is ample room for more learning in this dynamic media landscape.

For funders especially, there are potential collaborations whereby we could make moves to standardize our approach to impact. For example, multiple funders could consider using the same impact metrics across grantees. Together, we could share investments in baseline data, understanding where the key conversations are now as an initial step in defining the kinds of change we want to see. Finally, there is the opportunity to apply these efforts outside of journalism—what does it mean for advocacy efforts? Internal communications? How do we improve efficacy of social change organizations to communicate?

As nontraditional media funders, we appreciate that trust and communication with our partners is even more important when making investments without editorial control. In journalism grantmaking, we’ve found that it is essential to have a clear, well-articulated set of end goals and opportunities for learning along the way. Our partnerships with journalism organizations such as the Society of Environmental Journalists, Chalkbeat in the K-12 education arena, and others have shown us that relationships matter tremendously—and a two-way dialogue about what is working, what isn’t and what changes are made along the way is critical to the field, as a whole, moving forward.