Common Myths of Evaluation
May 23, 2006 at 9:57 am 4 comments
Looking further into outcome evaluation in different fields, I came across a very good resource with a rather quaint title: The Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources. Interesting to note the similiarities between my own top ten excuses for not evaluating and the common myths about evaluation that the author lists; as follows:
Myth: Evaluation is a complex science. I don’t have time to learn it! No! It’s a practical activity. If you can run an organization, you can surely implement an evaluation process!
Myth: It’s an event to get over with and then move on! No! Outcomes evaluation is an ongoing process. It takes months to develop, test and polish — however, many of the activities required to carry out outcomes evaluation are activities that you're either already doing or you should be doing. Read on …
Myth: Evaluation is a whole new set of activities – we don’t have the resources No! Most of these activities in the outcomes evaluation process are normal management activities that need to be carried out anyway in order to evolve your organization to the next level.
Myth: There’s a "right" way to do outcomes evaluation. What if I don’t get it right? No! Each outcomes evaluation process is somewhat different, depending on the needs and nature of the nonprofit organization and its programs. Consequently, each nonprofit is the "expert" at their outcomes plan. Therefore, start simple, but start and learn as you go along in your outcomes planning and implementation.
Myth: Funders will accept or reject my outcomes plan No! Enlightened funders will (at least, should?) work with you, for example, to polish your outcomes, indicators and outcomes targets. Especially if your's is a new nonprofit and/or new program, then you very likely will need some help — and time — to develop and polish your outcomes plan.
Myth: I always know what my clients need – I don't need outcomes evaluation to tell me if I'm really meeting the needs of my clients or not You don’t always know what you don’t know about the needs of your clients – outcomes evaluation helps ensure that you always know the needs of your clients. Outcomes evaluation sets up structures in your organization so that you and your organization are very likely always focused on the current needs of your clients. Also, you won’t always be around – outcomes help ensure that your organization is always focused on the most appropriate, current needs of clients even after you've left your organization.
Glenn
Entry filed under: Evaluation tools (surveys, interviews..), PR evaluation, Training evaluation.
intelligent measurement 
1.
Nope | May 23, 2006 at 1:26 pm
I know it’s how you make your living an’ all, but just as a little evaluation, would you like to give us the complete, non-mytholigised, referenced facts comparing the success of evaluated campaigns with the success of non-evaluated campaigns?
This will of course be so completely definitive and persuasive it will inevitably illustrate how paying money for measurement makes a lot of economic sense, and is not really just a way of selling another ‘service’. Won’t it?
And then no one will ever need convincing of the value of spending their comms resource on measurement, not comms, ever again.
2.
Glenn | May 24, 2006 at 12:50 pm
Dear Nope,
That’s an interesting perspective – but how do you judge the success of a campaign? You have to make some sort of judgement – evaluation perhaps?
I am not claiming that evaluation itself makes a successful campaign, but my experience has been that evaluation assists us in understanding if we achieved what we set out to do.
Also evaluation does not have to be expensive and should be integrated into the tasks of any communications professional.
Glenn
3.
Nope | May 24, 2006 at 9:20 pm
Thats for your compliment on the interest value of my perspective Glenn.
I’m still wondering though, if evaluation is worthwhile, wouldn’t it be easy to show evaluated campaigns are more successful?
Or, if the evaluation assists us in understanding if we acheived what we set out to do… wouldn’t it be possible to put some hard numbers on how much value there is in possessing that understanding?
Wouldn’t that be an interesting perspective?
All that evaluating…there must be some pretty compelling numbers around.
4.
Glenn | May 26, 2006 at 8:45 pm
That is an interesting proposition – to try and put some hard numbers on how much we value the understanding we gained from an evaluation.
We could imagine from an evaluation that we learn that a certain communication channel is no longer reaching a given target audience. Therefore, we abandon that channel and focus on the most efficient channels. We could certainly put numbers on the cost saved by abandoning the inefficient channel.
In certain evaluations, this would be possible. And it makes you wonder, has anyone done it already?
Glenn