Idealistics is shutting down. This blog will continue to be updated at fullcontactphilanthropy.com.
If you had to choose between an evaluator or a marketer

Earlier this week I asked the following question on Twitter

If you had to choose between an evaluator or a marketer for your #nonprofit org, which would you pick and why?

I had plenty of interest in the question, but only one answer. Ann Emery pointed to a 2010 study by the Innovation Network about the State of Evaluation in 2010. The salient point from the report that Ann pointed out is that in an online survey of over 800 non-profit organizations across the United States, “fundraising is #1 priority in nonprofits while evaluation is #9…”.

This statistic provides some support for the open secret that organizations prefer investing in marketing over evaluation, but it doesn’t answer the second part of my original question, why do organizations choose marketing over evaluation?

While not intentionally an answer to this question, a nonprofit I have been almost working with for the last year provides some insight. This organization is relatively young, small nonprofit that has found itself a media darling in certain circles. It plays lip service to a desire to evaluate its findings but insists that anyone who look at their numbers be properly vetted (read already a believer in this agency’s approach).

Evaluators are inquisitive, and skeptical, by nature. A hypothesis test assumes there is no effect, rejecting this assumption only in the face of convincing evidence. Evaluators do the same thing.

This organization (not-uniquely) starts from the standpoint that its impact is a given. In that mindset, evaluators can only disprove your asserted greatness. Thinking of it that way, I’m not sure I’d hire an evaluator either.

An investment in marketing however brings accolades from the press, photo ops with politicians and the adoration (and financial support) of the general public. So really a choice between marketing and evaluation is a choice between fame and fortune versus the possibility of uncovering that the project you have invested in for over half a decade doesn’t do what you thought it did.

In this way, choosing the marketing consultant is the only rational choice to make. Well, that is, if your organization’s logic model defines an ultimate goal of self aggrandizement. If instead your target population are the people your agencies aims to serve, and the impact theory defines causal linkages between your interventions and something other than coverage in the New York Times, then an evaluator might be an okay idea after all.

Consumer protections and false advertising

Corporations invest in evaluating their products in part because better products are more competitive in the market place, but given the indirect funding nature of nonprofits, where the service recipient is not the purchaser of services, this incentive falls apart.

However, corporations also evaluate their products as to not run afoul of the various consumer protection regulations placed on businesses, including laws against falsely advertising one’s products effects.

Imagine how different the sector would look if a similar standard were applied in the social sector. I have written before that evaluation brings truth in advertising to the social sector, but the real benefit would be to those we serve. The media story should be a secondary bi-product of a job well done. Instead, getting a good media story is the job, period.

  • Betsy

    Do nonprofits really hire evaluators to answer a question to which they don’t already know the answer?  Yes, sometimes evaluations help nonprofit programs answer tough, critical questions – but I think more and more nonprofits are asking them, and their boards are, too.

    Most importantly, more and more funders are requiring nonprofits to ask and answer these questions.  So I’m not sure I agree that choosing marketing is a rational choice anymore. I can think of a couple of explanations.
    Maybe it is a choice based on cost? Evaluation done well, I suspect, is more expensive than a small marketing campaign.  How would the sector look if evaluations were similarly priced to a marketing campaign? Or if resources to do evaluation were less scarce? Maybe it is the assumption that evaluations are only done to judge the worth of a program? Your perspective assumes that the only role that evaluation plays is to be judgmental of the intervention, when in fact there are whole branches of evaluation that focus on developing programs, improving processes and informing the growth and impact that an organization can have.  It’s unfortunate to see evaluation portrayed in your light – it’s a challenge I find myself fighting every day, especially as an internal evaluator. The evaluation firms with which I am best acquainted promote the right approach for the organization and where it is developmentally.  At least we agree that evaluation is about investing in your product, making sure we are as true as we can be in touting our benefits int he social sector.

    • http://www.fullcontactphilanthropy.com David Henderson

      I Betsy, I’m afraid you have misconstrued my argument. Choosing marketing over evaluation is rational only if your decision calculus assumes that the only value of evaluation is to disprove one’s prior assumptions of impact. This is a calculus perpetuated by a misunderstanding of the real value of evaluative metrics.

      To that end, I’m not sure lower cost evaluations would necessarily get the result you and I both desire. If the only reason agencies do not invest in evaluation is cost, then sure, but such thinking ignores the evidence, which suggests evaluation is a relatively low priority, especially if one assumes evaluation bears more risk than reward.

      • Betsy

        David, I think your argument does short shrift to helping people understand there are many purposes to evaluation beyond proving impact.  There’s a long cycle in programs to get to impact, and sometimes evaluation is part of that process.  And I by no means am implying that “lower cost” evaluations are the aim – but the reality is that evaluation can be prohibitively expensive to be done well.