The pressures on all of us to filter stories and visual media shouldn’t mean that we’re filtering it out. At my firm, See Change, Inc., we believe the social sector has as much to gain from becoming expert consumers of stories and visual media as it does from refining impact metrics and establishing performance management systems. Metrics can indicate whether or not, and to what extent an intervention is affecting a target population. But stories are still the best method we have for understanding how an intervention works, how and where to replicate it, and how it can scale and become sustainable – all critical questions as we seek real impact.

The first step in becoming better consumers of visual media in the social sector is to create a framework for sorting out types of stories. Was this story or this video created for fundraising or social marketing purposes? Is it designed to mobilize a constituency to action? Is it part of a body of systematic qualitative research? At the end of last month, we hosted a conference, inVision 2010, using these three topical strands to organize presenters’ sessions. Within the “visual media as research and evaluation” track, here are some of the key questions that social sector practitioners should keep in mind when presented with visual stories as evidence of change:

• The Anecdote Test: Is this story representative of outcomes frequently and reliably attained by the intervention, or is it an outlier? For example, if it’s the story of one individual whose life is better because of a program (a very common visual storytelling arc), what information, if any, is also available about other people in the program? How sustainable is the change depicted? Is this one positive moment in a longer story of ups and downs? What is the context surrounding this story of change?

• The Special Sauce Test: Does the visual story simply document one or a few examples of positive outcomes, and/or does it also delve into how that change was created? Documenting the existence of positive outcomes at a meaningful scale is usually accomplished more efficiently through counting – either through surveys or interviews – than visual storytelling. However, uncovering the best practices or special sauce of an initiative or program model – or the reasons why it did not create outcomes – is very effectively done by capturing stories directly from participants and having them explain the change in their own words.

• The Theory of Change Test: How was the storyboard for this piece of visual media created? Visual stories don’t come out of nowhere; they are carefully produced. Who authored the storyboard? What decision-making process was employed to select this story for telling? What other stories were available, but not selected, and why? Was the storyboarding process guided by a theory of change for the overall program or initiative? Does this story support or challenge that theory of change? Does it explore the theory of change fully enough? What other stories also need to be told?

There is both art and science involved in good storytelling in the social sector. I have no doubt about the ability of talented media makers to create compelling audiovisual art, nor about our capacity as human beings to be moved by powerful stories. We should welcome and celebrate those capacities in the social sector – emotion and empathy are valid drivers of our work. But I urge us all to get better at our “story science” as we journey toward social impact. Metrics alone are not enough.

(Photo by Carbon Arc)