By Kimberly Sevcik | Director, International Engagement, ITVS
Many of us who work in media believe that social documentaries and storytelling can change hearts and minds and even behavior—and with the field’s growing emphasis on impact measurement, Independent Television Service (ITVS) set out to prove it.
Women and Girls Lead Global—a multi-country project that used documentaries to move the needle on gender equality—provided ITVS with an opportunity to commission one of the most extensive studies ever on documentary film in the context of global development: five years, five countries, 34 films, 45,000 participants and dozens of change indicators. (Go to womenandgirlslead.org for a comprehensive list of the films making an impact around the world.)
The Aspen Planning and Evaluation Program monitored, evaluated and conducted the research for the project—a collaboration between ITVS, USAID, the Ford Foundation, and the Wyncote Foundation, with additional evaluation support from the Bill and Melinda Gates Foundation.
We set out to learn whether documentary films that are not custom-designed by social change professionals or strategic communications firms, but rather artful films made by independent storytellers and supported by ITVS—promote changes in knowledge, attitude and behavior?
The answer, in short, is yes. Across all five countries, the Aspen study found a consistent trend of positive change, with the majority of indicators moving 15 to 30 percentage points in the desired directions. The full report provides plenty of detail on which indicators shifted, and by how much. We’ve also compiled insights on why we think the project worked—and what things we might change next time around.
First, a bit more context: Working in India, Bangladesh, Jordan, Kenya and Peru, Women and Girls Lead Global partnered with established NGOs doing exemplary gender equality work on the ground. We collaborated with them to design campaigns using film to start breakthrough conversations that led to community-driven action. Each campaign was customized to the culture and the gender issue, and a few programmatic pillars anchored the project across all of the countries—among them, the three-film model.
The three-film model activates a series of facilitated film screenings for the same audience over a limited period of time. It allows each conversation to build upon the last one, providing cumulative learnings as well as opportunities to follow through on calls to action.
The purpose of the first screening was to convene an audience for a breakthrough conversation on sensitive issues that emerged during the film—child marriage, for example, or gender-based violence.
The second screening, using a different but thematically aligned film, primed the audience for calls to action. In Bangladesh, where we worked to keep girls in school and out of marriage by creating safer school environments, the call to action encouraged students to start an anonymous complaint box where girls could report harassment. In Jordan, where we aimed to break the silence around gender-based violence, NGO partners encouraged audiences to offer a compassionate ear to a survivor of violence.
The third and final screening was used to reflect on the actions taken by individual audience members—what worked well, what didn’t—and to elicit community ideas for collective action. In Peru, a group of young people decided to produce a music video urging other young people to seek out information about sexual and reproductive health. In a community in India, a group of young men lobbied the village council to start a women-only bus for female college students, who were dropping out of school due to sexual harassment on public transportation.
These collective actions underscore one of the great values of using film as a social change tool: It opens up conversations that empower communities to come up with their own solutions to challenges and supports change at the local level.
And that leads to the next lesson learned: Good facilitation matters. The Women and Girls Lead model prioritized the training of staff members at our NGO partners. They learned how to create a safe space for community members to engage in sensitive conversations and to lead them from discussion into action. The strength of our partner communities’ engagement and action plans were correlated to the knowledge, confidence and skill of their facilitator. Our takeaway? Develop excellent facilitator toolkits and invest in facilitator training.
In designing the project, we knew we would have to honestly assess what documentaries do well. And to develop a theory of change, and an evaluation framework, that took their limitations into account.
The documentaries we curated for Women and Girls Lead Global weren’t designed as precision instruments to instigate change. They were made by independent filmmakers to tell powerful stories about compelling characters—ideas that came from the ground up from around the world.
Studies from MIT and USC on narrative’s effect on the brain suggest that film can work as a mirror, reflecting to the community challenges similar to the ones they’re facing and increasing self-awareness. They also suggest that stories serve as a bridge, linking audience members to the struggles of people who are different from them, and seeding empathy.
Self-awareness and empathy are critical markers on the road to change, but they probably won’t overturn deep-rooted social norms around child marriage or gender-based violence overnight. Aspen collaborated with ITVS to design an evaluation framework that recognized the type of incremental changes that documentaries were likely to promote.
Women and Girls Lead and our community partners mapped out the change indicators that would eventually lead to bigger shifts on these issues. For example, women being aware of the laws protecting them against gender-based violence in Jordan; or men developing respect for women as one step along the path to preventing gender-based violence in India.
Aspen then tracked dozens of these indicators of incremental change, which were connected to 21 key objectives.
Among some of the promising findings:
- In India, where we worked with men to prevent gender-based violence, we saw a 32 percent decrease in men who felt they should have the final word on household decisions.
- In Jordan, where we sought to mobilize men and women to take action against violence and harassment, there was a 26 percent increase in audience members who said they would intervene if they saw a girl being sexually harassed.
- In Kenya, our goal was to get more women into political office. There, we saw a 43 percent increase in men who disagree that men are better suited than women to run for office.
- In Peru, our goal was to decrease early unplanned pregnancy; we saw a 40 percent decrease in adolescents who felt unprepared to discuss reproductive health with their parents.
- In Bangladesh, the goal was to keep girls in high school and out of marriage. In just two years, the child marriage rate decreased by 75 percent in partner schools. If the intervention were to be extended to all of Bangladesh, this represents more than 700,000 girls saved from child marriage per year.
Most of this quantitative data were gathered via traditional evaluation instruments: baseline and endline studies, focus groups and, in Bangladesh, a quasi-experimental study. But we also had the opportunity during Women and Girls Lead Global to develop and pilot DocSCALE, a new digital evaluation platform that enabled us to use a simple mobile phone survey to gather rich qualitative data—and to streamline and structure it. (For a brief summary of the principles that DocSCALE is built on, and how it works, read our article in Stanford Social Innovation Review. If you’d like to dive deeper, you can read about our pilot test results in this white paper.)
Our hope is that this new tool, and the lessons we learned about how to effectively use documentaries to create social change—and track their progress—will open the door for more social impact media interventions that harness the power of storytelling.