Editor’s note: Melanie Moore Kubo is the founder of See Change Inc., a consulting firm that specializes in helping non-profit organizations tell their stories of client progress in visually compelling yet qualitatively sound ways. We are pleased to have Melanie join us on FCP.

The current movement to make philanthropy more accountable for lasting social change is critical and must take deep root in our sector. Calls for an impact-oriented social capital marketplace have switched on a bright light in a dark room, urging us to look at all sides of our philanthropic activity – far beyond the honeymoon stage of the initial grant or investment all the way to the “so what” stage, when we may learn what difference any of our actions have made. Of course, philanthropists have used evaluation to examine “so what” questions for a long time, but a new vanguard – consisting of both next-gen social entrepreneurs and social sector veterans – are reconsidering and reframing them.

Our sector is buzzing with efforts to define and implement metrics systems that will support strategic philanthropy and impact investing. Getting better at using metrics is a vital evolution of our social change efforts. And yet, I worry when the conversation about metrics leaves out an essential piece of the puzzle – qualitative methods.

“What?!” you say. “Qualitative methods? I thought this was a serious post about measurement and accountability!” Indeed it is, and I’m grateful that David, who deals more in numbers himself, has invited me to contribute my thoughts on the use of qualitative methodologies. Please read on.

Social Change is Complex

For over 15 years I’ve designed and implemented mixed-method studies of increasingly complex social sector research questions. It used to be that funders wanted to know whether or not a program was producing positive outcomes in participants. Many still want to know this, but in addition, they want to understand more systemic issues. Here are a few examples of questions my team is currently examining:

• How does the collective capacity of many community-based organizations form a nationwide social justice infrastructure?
• To address the achievement gap between white children and children of color, do we need to look for solutions not just inside, but alongside the public education system?
• How can international development programs be sustained by local leaders once aid dollars have stopped flowing?

The field’s growing awareness of interdependencies has led to more meaningful inquiries into the impact of philanthropy and social investing. And the more complex the research questions, the more of a role there is for high quality, systematic qualitative research in addition to measurement of those things we can quantify.

Anecdotes are Not Qualitative Research

High quality, systematic qualitative research is not the same thing as collecting a few anecdotes and photos to put in the sidebar of a report. It is also not the same thing as fielding a written survey asking respondents about attitude or behavior change. I recently listened to a webinar that billed itself as being about “qualitative methods,” and the central discussion was about quantitative techniques for analyzing survey data (for example, running a statistical regression on data collected using 5-point Likert scales). Both definitions of qualitative research would make any self-respecting ethnographer cringe. In my book, both of these approaches – anecdote or self-report survey – reflect a very limited understanding of the range and power of systematic qualitative research.

In future posts, I’ll delve into qualitative methods increasingly used in the field for both data collection and reporting, including visual storytelling, social network analysis, discourse analysis, and visual arrays of massive data sets. I look forward to generating a dialogue with all of you about the productive uses of qualitative methods to advance our commonly held goal of making the world a better place.

(Photo by luz)