Idealistics is shutting down. This blog will continue to be updated at fullcontactphilanthropy.com.
Funder mandated outcomes requirements create perverse incentives for implementing organizations

On face, funding agencies providing grants contingent on implementing organizations meeting outcomes objectives seems sensible. After all, the sector is moving toward an era of smarter philanthropy and impact oriented giving, so shouldn’t funders demand outcomes for their money?

Kind of.

I have been aware of this problem for some time, but was recently faced with an ethical dilemma when a customer asked me to help them adjust their program offerings to better achieve funder required outcomes.

The organization I was working with provides employment services to low-income individuals. Their grant stipulated that a certain number of program participants had to get placed in jobs in a given period of time. At first glance this seems to be a simple optimization problem where the employment program wants to maximize the number of people placed into employment.

Given this simplistic directive (place as many people as possible into employment), the optimization is actually quite trivial; serve the people most likely to get employment and ignore the hard to serve.

In this case, the hard to serve also tend to be those who need employment assistance the most. People with children might be harder to place into employment, given childcare needs, yet these individuals arguably need employment more than the otherwise equivalent person without dependent children.

Similarly, better educated people are easier to place into jobs than those with less education. But better educated people are more likely to find employment irrespective of the program intervention. This, of course, highlights the difference between outcomes and impact. The grant required improvements in outcomes, that is, the number of people placed into employment. When the focus is on outcomes, simple rationality dictates finding the most well-off persons who still qualify for services, and serving those folks first at the detriment of those who are harder to serve but in greater need.

While serving those who are better off first is rational given the way outcomes thresholds are typically written in grants, it probably does not lead to the desired outcome of the implementing organization (nor the intended outcome of the funder either!).

Hence my firm’s ethical dilemma. By wanting our assistance in optimizing outcomes according to my client’s funder’s guidelines, our client was unwittingly asking us to help them identify and serve only those who didn’t need their help that much to begin with. Our mission is to help organizations use metrics to help people better, and in this case, a data oriented approach given a misguided objective would likely lead to under-serving a hurting demographic.

Of course, this mess is nothing new, and is outside the control of implementing organizations. Funders requiring meaningless metrics of implementing organizations is not news. However, as funders try to press their grantees for more results, I am concerned that funders with a poor grasp of the difference between outcomes and impact, and insufficient knowledge to properly operationalize social indicators, will force implementing agencies to act in financially rational ways that end up hurting their target populations.

The answer to this problem is better data literacy in the social sector. My practice to date has focused on the data literacy of implementing agencies, but I’m worried that the zeal for more proof of social impact has underscored an open secret; both our front-line and grant-making institutions have a limited capacity to use data effectively.

And to those who say that some data is better than no data, I would argue that data is more like fire than we tend to realize. Fire has done incredible things for humanity, but those who do not know how to use it are likely to burn themselves.

  • David Lynn

    Definitely agree with you on the need for data literacy (and to have the underlying data). However, while there clearly is a need for funder education, there is also a need for the nonprofits received the grant to ensure mission alignment.  Too often nonprofits take grants that force them into an area that may not be best for them.  In your case above, it appears on the surface that the mission of the nonprofit and the funder is not in line.  Perhaps the funder’s mission is purely about employment, maybe their entire mission is to reduce the unemployment rate.  In that case, their money is doing exactly what it should: get the highest number of people employed as possible.  Your client’s mission is to help the low-income population.  Those are not incompatible missions, but as you noted, there is room for plenty of differences in how to achieve those missions.  Thus, seems like the appropriate first step of consultation is not to help the nonprofit adjust their outcomes, but to discuss whether the missions are aligned – and with the funder in the room, if possible.

    • http://www.fullcontactphilanthropy.com David Henderson

      You raise a point worth considering, particularly the ongoing risk of mission creep as front-line agencies look for funding to expand, only to expand beyond their mission and becoming contracted implementors of the disparate missions of various funders.

      To the specifics of this case, the mandate was not to employ just anyone, but anyone below a given income threshold. Your argument then is that the funder is indifferent between someone one dollar and a thousand dollars below the set income threshold, which is fair. However, the underlying risk here, even if the funder is indifferent to which of those qualified for services is assisted, is that those closer to the income threshold might be less likely to have found employment on account of the intervention.

      While the funder might be indifferent between any two people eligible for the program, presumably the funder is not indifferent between a positive outcome caused by the intervention (impact) and a positive outcome not caused by the impact. The rational optimization under this funders decision calculus runs a greater risk of serving those for whom credit can be wrongly applied to the intervention.

  • http://www.facebook.com/profile.php?id=515921890 Neil Bakshi

    Is it possible that the solution might also have to do with translation between the org and the funder? Looking at the problem as a system, the interchange of information between the org and the funder is the fundamental connection. You can change how the org works and change how the funder works, but maybe the connection also has to be changed. Perhaps an intermediate node needs to be created that can help adjust.

    A more useful way of saying this is, the most difficult thing to explain to someone is why the things they want aren’t possible. Or aren’t a good idea. And it might be uncomfortable for the org to take that position, because they have mouths to feed. So maybe there needs to be someone in the middle whose job is specifically to make that translation. Perhaps in a consulting capacity? Or maybe as a distinct mission for a new org?

    • http://www.fullcontactphilanthropy.com David Henderson

      The funder sets the rules for the engagement in the grant requirements, a grant that is often one via competitive bid given a set of objectives rather than negotiated. In this way, I think David Lynn’s comment above is on the right track, that the implementing organization might have received funding from an organization with a mission different than its own.

      I think the greater issue here than two agencies whose missions are misaligned (as I actually believe this funder and implementing agency share the same mission) is that the funding requirements are at odds with both organizations’ missions.

      I would agree that there needs to be more thoughtfulness in examining the incentives funding requirements create, although I’m not sure I would advocate a middle-man intermediary between funder and recipient as a long-term solution. The problem fundamentally is one of nonstrategic (and dangerous) uses of information that, while in the name of ensuring outcomes per dollar, might do more harm than good.

  • http://twitter.com/isaac_outcomes Isaac Castillo

    Great post David.  Some of my quick thoughts on this.  

    1)  I think the funder and the service provider both share some of the burden on making this situation better.   The service provider needs to be true to its mission and population and provide services that create benefit and not just meeting reporting requirements.  The funder needs to do a better job articulating exactly who they want their grantees to serve with their funds – in other words, the funder should stipulate some sort of baseline criteria for those to be served, and then ask for employment placement outcomes for those participants. 

    2)  Of course, this doesn’t help with your short term problem:  what should the organization do.   The reality of the situation may mean that for the term of the grant the organization may need to do a mixed strategy – serve a certain amount of ‘low-need’ participants to get good outcomes to report, but then also serve some ‘high-need’ populations that will get placed in jobs at a lower level (or take longer to place).   The organization can then use both sets of data to engage the funder in deeper conversations.   In my experience, this the best way to bring about funder change.  

    3)  Even doing all of this, you may not be able to determine ‘impact’ but you may get to more appropriate outcomes for the intended service populations – which would certainly be an improvement.   

    • http://www.fullcontactphilanthropy.com David Henderson

      Thanks for the comment Isaac. I think you are right on points one and two, while point three (indeterminate impact) underscores the possibly perverse incentive for both funder and grantee. Even if the organizations mentioned in the post were able to better identify and measure outcomes, it is ultimately the outcomes measurement irrespective of impact that drives the incentive to focus on the “low-need” population.

      If our decision calculus is to maximize outcomes, and our outcome is binary, then we focus most on those who need very little help (our impact is marginal or non-existent) but we can claim credit for them. In this way, it seems a missing piece of the puzzle is wider understanding of the difference between impact and outcomes. I think the shift to being “outcomes oriented” is taking place, but what we really mean and strive for is an “impact oriented” sector. In part this situation illustrates the pitfalls of maximizing outcomes instead of maximizing impact.