I wonder however if the same logic doesn’t extend to the social sector. I work with non-profits big and small, all who claim they offer critical services. They make these claims to solicit donations, but in the aftermath of Too Big to Fail, the idea that certain social sector institutions are absolutely essential might be more unsettling than moving.

In the social sector we defend agencies’ rights to create monopolies, veiled in an avoidance of duplication of services. Social sector agencies argue that there are limited funds to produce social value with. Therefore, in order to maximize social output we should avoid situations where one agency’s service offering overlaps with another. The result is regional monopolies whereby a few organizations are granted full control over a range of social services, thus manufacturing an environment where agencies can claim to be critical backbones of the social sector.

Their claims of being too big, too critical to society, to fail, are legitimated by the dogmatic adherence to avoiding duplication of services. All the praise heaped on these organizations amazingly echoes the same complaints we now have about unwieldy financial institutions. Anti-competitive social sector collusion is the backdrop for the social sector version of Too Big to Fail.

The idea of social sector organizations being too big to fail is even more disturbing considering we don’t have much reason to believe they are effective, even though their entire hegemonic reign is predicated on the assumption of maximum social impact.

Organizations that successfully convince the public they provide essential services might very well be producing more marketing miracles than Hallmark moments. In fact, I have serious doubts about most organizations’ capacities to measure changes in client indicators period, let alone asses the extent to which those changes are the result of program activities.

I recently read a report about program measurement released by the Gates Foundation titled A Guide to Actionable Measurement (non-profit consultant Gayle Gifford has a nice write up about the report on her blog). While the report provides a comprehensive evaluation strategy, I was struck by the simple advice the report offers to not always focus on measuring social impact by relating program outputs to client outcomes.

I often drink the social outcomes measurement Koolaid, but this Gates report got me thinking that  simply understanding whether clients who receive a particular service are better off or not is a logical evaluative starting point. However, the extent of our evaluation woes is so deep that even such a simple, summary data type assessment of client indicators is more elusive than it ought to be, a problem that is compounded by a social sector culture that favors non-profit monopolies.

Through monopolization a handful of organizations, rightly or wrongly, are allowed to claim credit for a community’s status quo. The anti-competitive nature of the social sector creates an atmosphere whereby every organization can claim that reduced funding or closure of their programs will lead to social disarray.

I view the problems of anti-competitiveness and poor evaluation in the social sector to be linked. Anti-competitiveness, and adherence to avoiding duplication of services, supersedes the need for serious evaluation. Evaluation is not only a tool for evaluating client progress, it is a lens through which comparison between interventions, and agencies, can be drawn.

So long as the social sector continues to espouse anti-competitiveness cloaked in the flag of collaboration, we will continue to have a sector that is not necessarily too big to fail, but is certainly too small-minded to succeed.

(Photo by jeffisageek)

To me, the answer to what makes an organization high achieving is clear.  The social service sector exists to reduce social ills like poverty, homelessness, and food insecurity.  Organizations that have a greater impact on improving the lives of their clients are better than those that have less.  Any evaluative framework that is not centered on measuring changes in client indicators is irrelevant. Despite this obvious point, I am dismayed by how celebrated efforts like the Alliance for Social Investing and Greatnonprofits.org fail to base their evaluative criterion on client outcomes.

There is a lot at stake in getting a rating system right (or wrong).  The potential harm a poor rating system can cause was illustrated last week in a partnership between Greatnonprofits.org and Guidestar.org.  These two rating organizations teamed up to compile a list of the “Top Ten Relief Organizations Working In Haiti.”  The list was compiled based on a handful of donor reviews, and as non-profit consultant Gayle Gifford pointed out, those organizations “that were listed in the Top 10, had ONLY 1 or 2 Reviews. That’s it.”

Greatnonprofits.org and Guidestar.org responded to Ms. Gifford’s criticism by dropping the top ten list all together.  While these rating organizations certainly did the right thing by retracting their list, it is amazing to me that two supposed evaluation leaders in our industry could have compiled such a hasty, pointless agency ranking in the first place. There is so much that is problematic here, least of all the paltry number of reviews the top ten list was based on.

If we are to ever develop a meaningful top ten list of the most effective social programs, we have to embrace the social scientific complexities of evaluating clients’ social outcomes.  This means taking the collection and analysis of client data, in its quantitative and qualitative forms, seriously. Simplistic rating systems that ask donors how they feel about a particular organization may seem seductive, but they could not be more beside the point in determining which organizations are best able to improve the lives of hurting people.  So long as we fail to move towards an evaluative framework that is centered on sound social outcomes practices, the only top ten list we can reliably compile is the “Top Ten Worst Ways to Rank Non-Profits.”

Originally posted on inforumusa.org

(Photo by surfspirit)

Christmas is a lot of things to a lot of people. To some it’s a reminder of how great they have it and that they should be more giving to those who have less.  Those who have more might have volunteered today, given away a turkey to a needy family, and felt great about it. Don’t get me wrong, I’m grateful to those who step up during the holidays to improve the lives of others, some people don’t care for anyone but themselves (I even dated one of them for a long time). Just because I am grateful to those who give does not mean that I like charity.

In order for there to be a giver, there must be a receiver. For there to be a hero, there must be a victim. Indeed the social service sector is one that exists to help those who are victims of the economy, physical violence, drugs and alcohol, mental illness, etc. I cannot relate to those in my sector who say they love charity.  How can anyone love this? I do what I do because I love people, but I don’t love charity.

So there you have it, I am a dedicated social servant who hates charity. Merry Christmas, here’s looking forward to a  future where your donations won’t be necessary.

(Photo by Funky64)

I don’t just think about the poverty status of others, I think about the privileges I have in life and how that contrasts to the people we serve.  At times I get external pressure to move out of the social service sector, that a career in social services is a “get poor quick” scheme.  So the saying goes that with the privileges I’ve had in life such as a good upbringing and a fantastic education, that I don’t belong in social services.

I had an interesting exchange the other day with a person who shall remain nameless, but is very dear to me.  She argued both that poverty is such a complex problem that it is insolvable, and that I am too bright and too well educated to work in social services.  I asked her “if I am so well educated, and so bright, then shouldn’t I use my talents on the most complex and important social issue we face as people?”

Making money is very doable.  Selling products to people who have everything, specializing in the happiness of people numbed by pampering and entertainment, we’re collectively good at that.  Providing opportunity to the one in five kids living in poverty, or the 50% of youth who will be on Food Stamps at some point in their childhood, these are things we suck at.

I am exactly where I belong.  In going exactly where I belong, I realize I have been to places I probably should never have been.  I have been in neighborhoods that people like me don’t frequent, befriended people who, due to class standing, should be invisible to me.  I work twice as hard as the people who would otherwise be my peers for a fraction of the pay.  And I really couldn’t care less.

I get the point, if I weren’t in social services I could have a nicer place, a nicer TV, and my ex-girlfriend.  I assure you, I don’t want any of it.  I am exactly where I belong.

(Photo by slipper buddha)

About a year ago I wrote about an organization, the Alliance for Effective Social Investing, which aims

To drive more funds to high performing nonprofit organizations by helping donors adopt sound social investing practices.

They plan to do this by creating an evaluation standard by which organizations can be compared to one another.  I recently wrote a post for Inforum where I provided an update on what the Alliance has accomplished (nothing).  Last night I had the displeasure of reading through the group’s most recent paper, Social Investment Risk Assessment Protocol, 11th Version.  The document provides a questionaire and framework for non-profit evaluators.  The idea is that if all evalutors use this assesment tool, then we will have common metrics.  There are two problems with this approach.

1. Inherently not scalable – it is a fantasy to think every organization can get independently evaluated in any meaningful way, with any regularity.  If we can’t do this to every organization, or even a reasonable fraction, there will be no common metrics because the number of evaluated organizations won’t be significant.
2. Subjectivity – the evaluation methodology proposed by the group is based on the subjectivity of the evaluator, rating organizations on a scale of one through five on issues like whether or not an organization holds staff accountable through performance reviews.

On Wall Street, companies are not invested in based on whether or not they have performance reviews.  Companies have performance reviews because it keeps productivity and innovation up.  Higher productivity and innovation means greater profits.  However, presence of performance reviews, in and of itself, is not meaningful.  For companies, they are evaluated in large part on their profits.

So what is the common currency by-which social service and non-profit organizations should be evaluated?  That is the central question, and the one that the Alliance completely fails to address.  The real point should be to evaluate what gets done, not how we do it.  The inadequacy of the Alliance’s approach is on their focus on the how.  In evaluation speak, we refer to this as focusing on outputs, what we do, rather than outcomes, what results we get for the people we serve.

A better common metric are client outcomes such as changes in poverty status, housing status, food insecurity, educational outcomes, etc.  It’s funny how evaluations are incredibly trendy to discuss right now, yet nothing is really being done to move the sector any closer to meaningful evaluation metrics.  So far, this is largely the case in both the domestic and international spaces.  While the Alliance, to date, is a non-factor in seriously providing evaluation frameworks, I’m interested now to see what the Acumen Fund does to move this issue forward with their highly anticipated Pulse evaluation system.

Part of the problem is that performing social service evaluations is complex and can be costly. Agencies don’t always have the expertise or funds to perform proper evaluations. The bigger problem, though, is that agencies don’t necessarily have an incentive to conduct evaluations in the first place because of a common and wrongheaded assumption that if a social service agency is doing a nice thing that sounds intuitive, then it must be working. This is a dangerous assumption that is perpetuated in media portrayals of social services.

Take for example a recent article in the LA Times about a pilot program in the LA County jail system to distribute condoms to inmates. The intended purpose of the program was to reduce the spread of HIV amongst inmates. On face, it sounds like a compelling idea. We know that the use of condoms reduces the spread of HIV and other STD’s in the general population.

However, the jail environment is different. There may be a stigma effect in the jail that reduces substantially the number of inmates willing to opt-in to the condom program. Also, since the pilot program only supplied a limited number of condoms, it’s possible that inmates reused the condoms, used them improperly, or even engaged in riskier sexual behavior on account of the presence of condoms.

A prison is a perfect place to do a meaningful evaluation. The jail has medical information on inmates and could easily track which inmates opted-in to the voluntary condom program. Also, since the pilot was not conducted at all LA County jails, the non-participating jails could act as control groups.
While the article mentions a group of health advocates who said

that a successful review of that program could lead to widespread distribution of condoms in prisons throughout the state

the evaluation is at best framed as a necessary formality to confirm what is believed to already be known. Instead of casting a critical eye on the effectiveness of the condom program at reducing HIV transmissions, the author implicitly presumes the effectiveness of the pilot program by failing to challenge at any point the assertion that the condom program reduces HIV transmissions amongst inmates.

My point here is not to say that the pilot was ineffective at reducing the spread of HIV. Rather, my point is that without an evaluation, we simply don’t know. If we want to get real about helping people, we have to know whether we really are helping people.

Instead, we have squandered this opportunity, complaining we are light on solutions because we are light on funding. But lack of money is not the reason we have failed in the social service sector to make significant gains in solving social problems. I have no doubt that if we actually knew how to solve social problems, that the American public would give us the support we need.

The reason we don’t have the money we want is because we do not have a track record of success. We have failed to succeed so far because of a collective culture that is anti-competitive, and is deathly afraid that if we started rigorously evaluating the services we provide, that we would discover we aren’t really doing anything.
Therefore, it’s not surprising that as an industry we fail to produce good ideas, considering that we don’t really think.

Other industries use professional journals and online forums to exchange ideas, think critically about what works and what doesn’t, and encourage competition so the best innovations rise to the top, and others get thrashed and left behind, as they should. We in social services however, spend our time patting ourselves on the back, telling one another how great we are, and sharing articles about how there are too many poor people and not enough social services.

It’s time for us to get real. Not every idea is a good one, not every agency has a right to exist, and even if we “keep working together” that does not mean “it is going to get better.” We need a cultural revolution in the social service industry. We need to start taking ourselves as seriously as the problems we work on. So long as we hold back on critiquing ourselves, and one another, our industry will continue on its path to mediocrity.

His book is testament to the power of moving outside of one’s class and highlighting the realities of poverty through first hand accounts. Venkatesh got me thinking about my own experience working in social services with low-income and homeless individuals and families. My experience as a service provider, volunteer, and agency executive has shaped my understanding of what works and what doesn’t in helping people lift themselves out of poverty.

I have not been a gang leader for a day, nor do I plan to be. But I have been a contributor to the social service sector for eight years, and have some stories worth recounting and insights worth sharing. I plan to do so in coming blog posts.

The irony of course is that in economic down turns the need for social services goes up, even though providers cut back. It seems every time we find ourselves in an economic crisis, we collectively lament that no one thought ahead and preserved money for a rainy day fund so that we would have services when people need them most.

So it got me thinking that there should be a foundation that only gives out grants in an economic crisis. An emergency fund for emergency services. This way, when vital social services find their budgets pinched, this foundation could step in and provide some monetary relief.

The foundation would fundraise like hell in good economic times and refuse to give away a dime. It would only give out grants during rough times. It’s would be kind of like a forced savings plan on the social services sector. Your thoughts?