Full Contact Philanthropy

Giving USA report suggests human services must do more with less

David Henderson August 2, 2011

About a month ago Giving USA released a report on charitable giving in 2010. The report found that on the whole charitable giving increased modestly in 2010 relative to the prior year.

While the philanthropic community received the report with guarded optimism, the report had decidedly bad news for organizations in the human services field. From 2009 to 2010, giving to human service organizations actually decreased 5.6%.

So while those concerned with philanthropy might have seen some silver lining, or perhaps down right good news in the numbers, from where I sit the findings were all bad.

You see, I’m not in the charitable sector, I’m in the ending poverty business. And while demand for social services have logically increased during a prolonged, hopefully not structural loss of American economic prowess, charitable giving to poverty focused organizations has dipped in tandem.

The funding environment is not going to improve any time soon, and if our economy does suffer a structural shift, it might not ever. Human services has never been the sexy cause of affluent philanthropists, who tend to favor health, arts, and higher education. Indeed, charitable giving in human services is largely driven by middle and lower-class households. Rounding out human services is government funding, a lot of it. Of course, that funding is being cut back substantially as governments slash budgets to stave off default.

The stark funding reality in the face of higher demand leaves only one choice. And no, that choice doesn’t have anything to do with $3 dollar Twitter challenges and a hollow promise of social media fundraising success.

We need to get real, and getting real means doubling down on achieving social impact. The only way to do more with less is to get more efficient. Efficiency, of course, cannot be discerned without reliable ways of measuring if what we do actually works.

Housing First, which advocates placing chronically homeless persons into housing rather than trying to treat them while living in the streets, caught on as a solution because it demonstrably houses hurting people while simultaneously decreasing the social cost of homelessness.

This is the kind of innovation we need to pursue, one that is driven by facts and inherently measurable. Sense we can measure the effectiveness of Housing First, we can regularly monitor whether the strategy actually makes social and economic sense. Discerning both the social and fiscal impact of any program must be a prerequisite, especially as funding gets increasingly more difficult to come by.

Ultimately, the changing economic landscape necessitates a shift in the way the social sector operates. I’m afraid the over-emphasis on online fundraising and corporate media challenges only serves to distract from the stark truth; it’s time we hunker down on identifying what works, and cutting what doesn’t.

Of course, if we are to focus our efforts only on solutions that work, we’ll need better ways of measuring success.

Measurement though, I’ve come to learn, has less to do with a solid set of equations and more to do with an organization’s culture. Historically, our sector has a culture of emphasizing increasing revenues over monitoring impact. In the current funding environment, that strategy no longer cuts it.

If our ultimate goal is to end poverty, we have to face the fact that growing the funding pot is not presently a strategy we can pursue. Instead, we will have to do more with less. In order to do that, we have to focus on what works, seriously evaluate our own efforts, and have the courage of our convictions to put the needs of suffering people above the monetary ambitions of our interventions of choice.

The benefits of quantifying uncertainty

Michael Gechter July 19, 2011

A couple of weeks ago, David wrote a piece on Poverty Insights criticizing the Los Angeles City Homeless Services Authority (LAHSA) for failing to report margins of error around its estimate of the size of the Los Angeles Homeless population.

Most likely, as David argued, LAHSA failed to report the statistical error bounds around its estimate because doing so would have revealed that LAHSA could not statistically determine whether homelessness had increased or decreased since its last estimate, collected in 2009.

LAHSA felt pressure to show that its efforts led to a decrease in homelessness and, therefore, reported a point estimate that did exactly that. In so doing, the organization robbed homeless services providers of valuable data that can be used to provide better programming.

The reality is that reporting 100% accurate statistics on social phenomena is extremely difficult and is only advisable if the planning benefits outweigh the data collection costs. But quantifying our uncertainty can be useful.

In his book, “Identification for Prediction and Decision”, Charles Manski gives a relevant example. He refers to a study that randomly sampled members of the shelter population in Minneapolis one winter.

The study’s objective was to learn the probability that a randomly selected member of the homeless population would have secured housing six months later. Unfortunately, when it came time to do follow-up interviews, the researchers could only locate 64 of the 106 men originally sampled.

The researchers had no idea whether the 42 men who could not be located had exited homelessness or not. Given the situation, what could we say about probability of exiting homelessness?

Just like the exact number of homeless persons living in Los Angeles county in 2011, we can’t say what that probability is for sure. But we do know that the probability of exiting homelessness would be the lowest possible if none of the 42 men secured housing.

And the probability would be highest if all 42 men exited homelessness. Combined with the knowledge that 21 of the 64 men who were located did leave homeless, we know with certainty that the true probability of a man in this sample exiting homelessness must lie between 20% and 59%.

That’s a pretty big range. Is it useful?

It can be. Subject to sampling error and the time frame of the study, a homeless services agency operating in this area can expect that, under the current service regime, 20% of its clients will have exited homelessness in six months. The maximum improvement that a new initiative can achieve is to ensure that the other 80% secure housing. Bearing this in mind, the agency can trade off the maximum potential impact with the cost of the new intervention.

If the agency decides that it needs a better estimate of the probability of securing housing in six months, it can commission a new study that works harder to locate sample members six months after the initial survey. But, if the original study only reported a point estimate between 20% and 59% (say 40%), as LAHSA did, our agency would not know that it needed a new study.

This example shows that even simple studies yielding very imprecise estimates can provide useful content for programming and decision making. In order to unleash that content, we have to take statistics out of the realm of propaganda, where the LAHSA study is firmly ensconced, and into the program planning discussion.

Network effects and social outcomes of the poor

David Henderson July 6, 2011

Growing up I didn’t care much for superhero stories. In order for there to be a hero there necessarily has to be a victim. I figured since no one really wants to be the victim, I should not aspire to be a hero.

Perhaps that is why I’ve never been entirely comfortable with the traditional community service model of the enabled case manager providing services to the debilitated service recipient. There are several inherent problems with this traditional framing, not least of which is that the model defines service recipients by their needs rather than assets. Just as it is better to define words by their properties rather than lack thereof, so too should all people be defined affirmatively.

In keeping with this intention of positive definition, one of the things that defines all people regardless of economic standing is our social networks. However, positive network effects do not fit neatly into the standard community service narrative. Instead, we ignore the possibility that positive network effects can co-exist in poverty, assuming the poor are better removed from their networks all together, perhaps in favor of inclusion in a more affluent, mixed-income community.

I recently read an excellent article by Katya Smyth, founder of the Full Frame Initiative, in the Foundation Review titled “Leveraging Social Networks in Direct Services”. Katya argues that service providers do their clients a disservice by trying to treat symptoms in isolation, without utilizing the benefits of the greater social context and networks in which all people operate.

Indeed, social outcomes might very well be enhanced were social networks leveraged, rather than eschewed, in service delivery. That is part of the premise behind the Family Independence Initiative, which promotes the benefits of poor families working together within their own defined networks to lift themselves, and one another, out of poverty.

These ideas are natural permutations of the lending circle concept pioneered in microcredit. Lending circles are not organized by the lenders, but instead by the borrowers who build their own networks to both ensure the borrowing group does not default, but also to establish a supportive network and pooled resources that financially benefit all participants.

While there are obvious untapped opportunities in utilizing the existing networks of those in poverty, so too should interventions aim to positively augment social circles where possible. In Katya’s article she notes that in professionalized social services as they exist today, a case manager is more likely to try to help an individual secure employment by sending out dozens of resumes to anonymous employers rather than leveraging existing contacts and relationships. In this case, the missed opportunity is not just a failure to rely on the social network of the job seeker, but more broadly the interventionist’s reluctance to extend her own network to those she serves.

Network augmentation is fundamentally at the core of Mark Horvath’s We Are Visible, which aims to connect unhoused persons with a broader community through social media. Mark has recently received some criticism that although his focus is on homelessness, many of the people who follow him on Twitter and partner in his efforts are not homeless themselves. Considering that the intent of his intervention is to empower unhoused persons with a broader social network, those criticism fall flat, as network strength is in part a function of network heterogeneity.

While the current thinking around social networks and the social sector is stuck in low-gear on simplistic social media advice for non-profits, the real power of networks is in helping people lift themselves out of poverty. My hope is that social media can catalyze wider thinking on the value of networks generally, as the true value of network effects is in the enumeration of poverty reduction, rather than number of followers of a Twitter account.

Evaluation is neither pro-funder nor anti-organization

David Henderson June 24, 2011

The social sector is abuzz with the bruhaha over the proper role evaluation should play. There are some who argue evaluation is all important, that every intervention is guilty until proven innocent.

Others, like Jonathan Lewis, have taken the mantel of evaluation skeptic, arguing that it is the intervention, not the evaluation, that is paramount.

Personally, I have grown tired of the evaluation debate. The more interesting question, not only to me but to those who (allegedly) benefit from the services of the social sector, is how can evaluation make interventions more effective?

To be clear, this is a different question than the core query driving the debate in philanthropy circles. Those who identify on the philanthropy side (which despite the title of this blog, I do not) think about evaluation as a tool for deciding what to invest in.

While evaluation can, and should, inform investment decisions, the brighter promise of evaluation is in informing the decisions of implementers. In this way, I think about evaluation in two tiers, one that provides shorter term ongoing feedback to interventionists and another that is longer-term, and more exhaustive in scope to prove or disprove impact.

Although this grander, longer term, up or down, live or die evaluation is seen by some as the sole objective of evaluation, the fact is we are a long ways off (if we will ever get there) of having such reliable binary indicators.

What we are equipped to do is develop interim, iterative indicators that can inform implementers of whether their interventions are headed in the right direction, and more importantly, how to adjust accordingly.

But in the current environment of philanthropist driven evaluation furor, imperfect evaluation techniques have been mistaken for reliable crystal balls. This has had a chilling effect on implementing agencies, as daring to use evaluation for tactical purposes creates the political liability that negative results can lead to loss of funding.

In the business world, investors invest in competent people with vision. Those people, the day to day managers, use a myriad of techniques to improve products and raise profits. One such tried and true technique is hiring consultants to evaluate any number of parts of their businesses.

Smart investors do not parachute in, scoop up an isolated internal memo, and use that to decide whether the company is going to hell or not.

Yet that is what we risk doing so long as we position evaluation as a threat to implementing organizations. ultimately, the issue is not how implementing organizations nor philanthropist funders feel about their investments or interventions, all that matters is what happens on behalf of those in need.

The best way to help people better is to foster an environment where seeking information and changing behaviors as a result is rewarded rather than de-funded.

Social Impact Bonds will fail without solving the evaluation problem first

David Henderson 11 comments April 4, 2011

Social Impact Bonds are the darling of the minute in the social sector. Another day, another push to pretend the social sector is a damsel in distress, only to be saved by the brawn and brilliance of public-sector innovations.

The problems Social Impact Bonds aim to address are real. There are limited financial resources available for social interventions, and we do need to figure out ways to maximize impact per dollar.

Social Impact Bonds attempt to do this by tying returns on investment to outcomes indicators. When an investor purchases a bond in a particular social program, she receives her principal plus an additional monetary return if the agency she invests in yields measurable results. If the agency fails to meet its outcomes threshold, the investor loses her money.

These bonds would be backed by the government, which would theoretically only be obligated to pay for successful social interventions. The intended benefits are

The government only pays for what works
New money enters the system as social impact bond investors seek profits
Non-profits are incentivized to achieve impact

However, it is not clear how different social impact bonds are from the status quo. Funders already require organizations to demonstrate their outcomes. It is unlikely investors are going to be able to do that any better than existing foundation and government grant review committees. Indeed, the real problem of course is ambiguous and non-uniform evaluation criterion, plus the general lack of capacity to monitor impact.

Furthermore, the argument that Social Impact Bonds will bring new money into the sector is an empirical point yet to be seen rather than a logical conclusion. Social investors already get financial benefits in the form of tax write-offs at zero risk. I’m not saying the risk/reward of Social Impact Bonds won’t work, but let’s not be so sure they will, either.

Ultimately, ideas like Social Impact Bonds and non-profit market exchanges that mirror the world of finance dance around, rather than address, the core issues facing the social sector. Investors are a necessary part of a functioning for-profit or non-profit sector, but innovation and execution is king.

We are so enamored of the giving and investing aspects of philanthropy, yet our eyes glaze when the charts and logic models come out. Designing investing models and website that connect us with our favorite causes are so exceptionally beside the point when we lack the cohesions and capacity to measure impact.

It’s no wonder those who focus on the giving side of philanthropy gloss over the implementation aspects. Actually helping people is hard, but giving sure feels great. So too, does making bold claims about how we should only give to effective organizations.

But what does effective mean, and how do we measure it? Those are the questions that actually matter, and without real answers, funding models are not going to make a lick of difference.

(Photo by expta.com)

Feeding the feedback loop

David Henderson March 31, 2011

In my last post I pledged to give up the cowardice comfort of chanting the mantra of why we should evaluate and instead move toward discussions of how.

If evaluation is to actually be a useful tool for organizations implementing social interventions, it is essential that evaluations are established in a feedback loop instead of conducted on a one-and-done basis.

Indeed, much of the tension between evaluation proponents and front-line organizations has been that organizations are forward looking, yet evaluations are historical by nature.

Figure 1: Evaluations look back, organizations look forward

Evaluations necessarily rely on historical indicators, with a general rule of thumb being the longer the time series the better. This focus on the past is at considerable odds with organizations that face not only the needs of their constituents today, but the always pressing concerns of how to meet their needs in the future.

Given this conflict, it is no wonder organizations are reluctant to dive into evaluations, especially those with more limited budgets. If today and tomorrow are your real concerns, why focus on the past?

You wouldn’t, not unless looking at the past could better inform tomorrow. That is where the feedback loop comes in.

A feedback loop is essentially a way of enabling an organization to learn from outcomes metrics on an iterative basis, both collecting and synthesizing social indicators with rigor and regularity. The following diagram shows one approach to a feedback loop, whereby an evaluation begins with an agency’s Theory of Change, followed by question design, data collection, then analysis.

Figure 2: An implementation of a feedback loop

After reviewing the analysis of information collected in the previous term, the loop then goes back to the Theory of Change. At this point an organization uses its feedback to reassess its approach, what is working? What is not? What can we do different? These changes are then reflected in altered program design and data collection questions and processes.

The idea here is to turn evaluation into a tool instead of a judgment (“you suck”). Good evaluation should not simply conclude with an up or down on a program’s effectiveness. Instead, evaluation should lead to options, several of them.

Figure 3: Evaluations should lead to several options

In this way, I see the role of the evaluation consultant to be that of a medical doctor, who not only diagnoses problems but recommends possible treatments. It is then up to the organization to decide what course to take, as they know their capacity and client base best.

Feedback loops are critically important, but they are difficult to establish and I cannot claim to necessarily do it right. My company’s approach is in part based around our database systems. Since we build the data collection systems for our customers we can help facilitate the entire question design and collection process.

Of course there are difficulties here. First, not all the data an evaluator wants actually gets collected, or at least not in the ways an evaluator would prefer. Moreover, getting social sector managers to agree that evaluation is important in their organization is one thing, but getting them into the habit of regularly re-evaluating their efforts is another.

Therefore, establishment of a feedback loop, and by extension good evaluation, is not only about what to measure and how to analyze it, but how to setup a managerial system and organizational culture receptive and capable of incorporating feedback.

(Photo by esti)

Shutting the door on the evaluation debate, time to focus on the how

David Henderson March 29, 2011

There seems to be reasonable consensus in several philanthropy circles that evaluation is important.

Impact investors want to know what programs work to better guide their charitable investments. Organizations designing and implementing social interventions want to know what works so they can better help people in need and earn donated dollars.

I am pleased to see our sector moving toward agreement on the importance of evaluation in the social sector. At this point though the conversation needs to shift from should we or shouldn’t we evaluate, to how the heck we are going to do this correctly. Therefore, I am no longer going to write about why we should evaluate, and instead will focus on how.

This second phase of the conversation is not only the most important, but it is by far the trickiest. When it comes to evaluation, two people can say the same thing (“measure impact!”) and mean two totally different things (for example “conduct a randomized control trial” versus “write a narrative about one of my superstar clients”).

Complicating matters further, there are multiple intervention types, focus areas, and evaluation techniques. Of course, perhaps the bigger problem is the general lack of evaluative insight and expertise, at least amongst philanthropy talking heads.

Indeed, the reason the evaluation debate has lingered so long on the “should we or shouldn’t we” is because, by and large, that is the beginning and the end of what a lot of proponents can say on the subject.

Before anyone argues my perceived evaluative insights are ballooning beyond my ability, let me be the first to take a needle to my own inflated head. I suck at this.

In fact, we all suck at this. I would suggest there are really two types of evaluation proponents:

Those who have the skills and opportunity to put their theories into practice
Those who have a WordPress blog and a desire to dump on implementing organizations in the name of philanthropic purity

I count myself fortunate to have the opportunity to not only write about evaluation, but to work with some amazing organizations every day that allow me to work through evaluation strategies that both work and fail in an attempt to improve the lives of their clients.

If we are going to get evaluation right, we have to get it wrong first. And just like there is pressure on organizations to discuss their failures, evaluators should discuss their failures as well. I plan to.

My interest in evaluation has less to do with impact investing and more to do with crafting effective social interventions. Good evaluation provides regular feedback, not simply on whether an intervention works or not (an unrealistically simplistic binary attribution to complicated social interventions), but instead on what parts of an intervention worked and why.

Regular feedback can make failing and succeeding interventions better. That is the real power of good evaluation and a feedback loop. In my next post I’ll further discuss the importance of feedback loops and the strategies and struggles I have faced setting them up.

What I won’t discuss though is whether or not evaluation is important. I think we all get the point on that.

(Photo by Jeremy Piehler)

Good evaluation habits start early

David Henderson March 22, 2011

It is common knowledge that bad habits are hard to break. The longer one lives with a bad habit, the more difficult it is to shake.

The same is true of social sector organizations. Good evaluation requires good habits. But good habits are difficult to form, and even more difficult to introduce into an organizational culture that has grown accustomed to evaluative misbehavior.

So first, what are good evaluation habits?

Broadly speaking, good habits involve

Intentional data collection designed to capture client indicators stemming from an organization’s theory of change
Regular, incremental program changes based on feedback from client indicators
Identifying comparison groups to compare progress against

Many organizations adhere to anything but good evaluation practices. It is especially difficult to get established organizations to change course, like turning around a cargo ship drifting downstream.

I have been hired time and again under the pretense of helping established organizations with their evaluation strategy. However, I have learned over time that while the request for evaluative assistance might be earnest, inevitably the conversation turns from “help us assess our impact” to “show us why we are awesome.”

I certainly understand why organizations devolve from the former to the latter. These established organizations have existing funders that want to see that their efforts have been paying social dividends. Ultimately, evaluators are brought in to prove to funders that money has been well spent rather than to help assess how future monies should be spent.

This makes sense, but it enforces bad habits. With funders breathing down agencies’ necks, agencies are incentivized to look for any positive trend in their data and report it as success, instead of looking for feedback that informs intervention design.

I’m not arguing that good evaluation habits are unachievable in larger organizations. But, instilling good habits in startup organizations is not only easier, it is also the low-hanging fruit in our collective shift toward a more outcomes oriented social sector.

The problem, though, has been that evaluation consultants (I stand amongst the guilty) have focused their efforts on snagging contracts with established organizations rather than working with emerging agencies.

To do our part in helping instill good habits in start-up social sector organizations, my company designed an evaluation consulting package that

Offers access to evaluation guidance year-round, as good evaluation strategies develop through time instead of in a single Word document
Fits in the budget of emerging organizations

Under the new package, we will work with organizations under three years old helping design and implement evaluation strategies. Because good strategies take time, we have structured our contracts to run annually with quarterly meetings and deliverables, instead of using an hourly rate model.

Each contract is a flat rate of $3,500 so startup agencies can afford the service and are not at risk of ballooning costs.

I love working in the social sector. Having the opportunity to help organizations develop effective evaluation strategies that inform program design is by far the most rewarding experience my team and I can have.

If we want to foster a sector that values the importance of good evaluation, we have to instill those principles in the fabric of the next generation of non-profit organizations, before they get stuck in their ways.

Logistics and liquidity: what the crisis in Japan teaches us about philanthropy

David Henderson March 16, 2011

The tragic crisis in Japan has been followed by an outpouring of sympathy around the world and frustratingly predictable financial opportunism by the non-profit sector.

Givewell and Good Intentions Are Not Enough have posted excellent pieces on why giving to aid organizations in Japan right now is not a good idea, despite our natural desire to do so.

In short, Japan doesn’t need it. Lack of money is not the issue in the rescue efforts. Nor should it be, Japan is one of the richest countries in the world.

However, like in Katrina, getting the logistic right has proven exceedingly difficult. Indeed, there are legitimate risks of starvation, dehydration, and death from airborne illnesses, not to mention the radiation risks. But these risks are not the result of financial deprivation.

Delivering services to people in need, like delivering any intervention, is tremendously complicated. There is a lot more to providing effective interventions than money. Given this fact, if improving peoples lives is really what the social sector is about, why so much attention on giving rather than doing?

Simple, the focus on giving is easier, rewarding, and for those who facilitate it, lucrative as well.

But the hard work of the social sector is in the doing. Figuring out what works and what does not. Sure, raising funds is important, but the crisis in Japan makes it clear that both good intentions, and ample financial resources, are not enough.

Yet heralded social enterprises like Causes are pushing people to give to a crisis that does not necessarily need the money. Why? Because that is all they can do.

And therein lies the lesson. I don’t mean to disparage innovations that facilitate giving. The problem instead is that the sector’s best talent focuses so much on donations.

Look at the technical prowess of Causes or the insightfullness of Givewell. The sector is lucky to have these organizations. But our most pressing problems are what to do, followed by how to finance, not the other-way around.

Raising money does not solve logistical issues, and increasing endowments does not necessarily beget higher impact social interventions. Is there a correlation between money and effectiveness? Absolutely. But money is hardly the sole independent variable.

I get why so much of the talent in the sector focuses on donations. If there is money to be made in the poverty sector, by and large it’s on the donor end. But if we want that money to go to actual use, we’ll have to do more than help organizations raise money and focus on criticizing interventions that don’t work. Instead, we’ll have to put our money, or at least the money of others, where our hearts are, and put our best talent on the front lines instead of the back channels of donor portals.

(Photo by un.org)

Subscribe

Recent Posts

Search