Don’t give me evidence, I’m pissed!
David Henderson     13 comments     February 24, 2010

It is no secret I believe evaluations need to be a central part of the work we do in the social sector, and that not all evaluative frameworks are created equal.  Certainly establishing a reliable system for measuring organizational impact eludes us, but even in the cases that we do have reliable outcomes metrics, I too often come across individuals and agencies who ignore sound data not because it is wrong, but rather, because it does not conform to their beliefs.

Those of us who work on poverty alleviation no doubt get into this field out of a commitment to reduce poverty.  As we join organizations we believe in or develop poverty interventions of our own however, our loyalties shift in subtle ways.  Where once the poor themselves were paramount in our ambitions, our ambitions instead become about the success of our solutions.  If our solutions do not reduce poverty, we are perversely incentivized to argue with the evidence before us, rather than admit what we are doing is not working.

In this way, our sector as a whole is not trying to end poverty per se.  Instead, we are trying to be the one’s to end poverty.  If we blind ourselves by ambition and moral outrage, we render ourselves incapable of allowing outcomes metrics and social indicators to guide our poverty interventions.  If we fail to use data, we will fail to end poverty.

(Photo by Piez)

  • Amy

    Mr. Henderson,

    Completely agree with you on the need for data. I would pose the question, however: are we really ending poverty? Or are we trying to create wealth? Would welcome your thoughts.

    @amycarolwolff

  • http://twitter.com/gotigermonk @gotigermonk

    David, Would you please give us some of the tools used to measure poverty; and those tools that do a better job at measuring?

    • http://intensedebate.com/people/david_henderson David Henderson

      Thanks for the question. The tools that do a better job are social scientific analytic techniques that address changes in indicators like educational attainment and income and assets over time. The problem with a lot of the techniques we use now is that they tend to be more organizationally focused, like on board strength, and when they focus on clients the analysis we use is more anecdotal in nature, or 'story telling', as fundraisers might label it.

      Although organizational capacity is important, and story telling is compelling for soliciting donations, neither captures the true effects of a poverty intervention. A good example of effective social outcomes analysis are the randomized evaluations being pioneered by development economist Esther Duflo. The problem with her approach is that it is not scalable as it is both highly resource intensive and requires a tremendous degree of sophistication.

      So, as you ask, what tools "do a better job at measuring" social outcomes? It's a good question, something I don't have the answer to, and lots of people are working on. The first step toward better tools, I believe, is in agreeing that the units of measurement must lie solely in changes in the lives of the poor, rather than changes in board makeup or an organization's endowment.

  • http://intensedebate.com/people/david_henderson David Henderson

    Hi Amy,

    Thanks for the comment. That is an interesting intellectual distinction, and indeed depending on the goal the strategies would be different. Income supplement programs like Food Stamps obviously are not about creating wealth while programs like micro-finance are. Based on the significant questions regarding micro-finance's abilities to create wealth long term, rather than simply smooth income, and the complete inability of U.S. welfare programs to lift people above the poverty line, I think the question is less about which one we are doing, and more so whether or not we are actually achieving either end.

    • Amy

      David,

      Agreed. But here's something else to consider. We talk about "lifting people above the poverty line"; "alleviating poverty"; what are we actually doing? I think there's a danger (just as there is in not using data) in using the "buzz words" that do not actually communicate the WHAT. For example- are food stamp programs alleviating poverty? Or are they providing access to food to those who don't have it? Why don't we keep it bottom-line, honest, and measurable- just as your call for data suggests?

      • http://intensedebate.com/people/delitzer delitzer

        I think the two things we're talking about here are poverty alleviation and poverty reduction. Poverty alleviation includes programs like food stamps, which attempt to blunt the effects of poverty. Poverty reduction includes job training and other programs that attempt to help people increase their income reduce the total number of people living below the poverty line. Some amount of poverty alleviation will always be needed as a social safety net for those who fall on hard times or are unable to care for themselves, but poverty reduction ultimately has the greater potential for impact on the lives of people currently living in poverty. When attempting to measure outcomes of poverty program, it's important to keep in mind which of these approaches they are intended to take.

      • http://intensedebate.com/people/david_henderson David Henderson

        Amy,

        Thanks for continuing the discussion, indeed we should be clear about what we are doing, although in the case of food stamps I'd argue it is not necessarily so obvious to say that it is a program that provides food to people who don't have it. One aim can indeed be to prevent hunger, but it can also be used to improve nutrition, or even to boost a family's overall purchasing power, so that money that would go toward food can go toward housing. That said, being explicit about what our programs are meant to do is important if they are to be effective. And if we aren't clear about what we are actually doing, we cannot measure whether what we do actually works.

        I'd be interested to get your more detailed thoughts on why you believe being explicit about what we are doing is valuable and the pitfalls of using "buzz words". Would you be interested in doing a guest post on the subject?

  • http://twitter.com/mwomack @mwomack

    This has been an interesting discussion and I will long remember to not simply try to be the one to end poverty!
    We at United Way are presently exploring programmatic as well as "community" measures for our work in family financial stability. Unemployment rate? Percent of 4-person families achieving self-sufficiency? Percent of people spending less than 40% of income on housing? Calculate our own financial stability index? My head hurts . . .

    I hope this discussion continues – I could use a few more brains!

    • http://intensedebate.com/people/david_henderson David Henderson

      Interesting to hear what United Way is doing around poverty, or wealth creation, as it has been suggested might be a better way of putting it. What standard are you all using for measuring self-sufficiency?

  • Pingback: Please, Stop Talking About The Social Entrepreneur – Full Contact Philanthropy

  • http://www.slideshare.net/dazzlewhiteproFREE/cholesterol-lowering-vitamins-on-slideshare Cholesterol Lowering Vitamins

    I usually don’t post in Blogs but your blog forced me to, amazing work.. beautiful. yea nice Work great more please ;) .

  • http://www.slideshare.net/dazzlewhiteproFREE/cholesterol-lower-on-slideshare Cholesterol Lower

    Hello webmaster I like your post.. I Really enjoyed your blog. I just bookmarked it. I am a regular visitor of your website I will share It with Yep. I lurk there often. You guys have a wonderful blog. Keep up the good work. more please fantastic :) .

  • http://www.slideshare.net/dazzlewhiteproFREE/carpal-tunnel-pain-after-surgery-on-slideshare Carpal Tunnel Pain After Surgery click

    Hey very nice blog!! Man .. Beautiful .. Amazing .. I will bookmark your blog and take the feeds also…. TY a ton for posting, it was very informative and helped tons. terrific fantastic fantastic.