Posts filed under ‘Campaign evaluation’
Workshop on communications evaluation
I recently conducted a one day training workshop for the staff of Gellis Communications on communications evaluation. We looked at several aspects including:
- How to evaluate communication programmes, products and campaigns;
- How to use the “theory of change” concept;
- Methods specific to communication evaluation including expert reviews, network mapping and tracking mechanisms;
- Options for reporting evaluation findings;
- Case studies and examples on all of the above.
Gellis Communications and myself are happy to share the presentation slides used during the workshop – just see below (these were combined with practical exercises – write to me if you would like copies)
Evaluating campaigns – new publication
An interesting new publication is available from the UK National Council for Voluntary Organisations (NCVO) on campaign evaluation. “Is your campaign making a difference?” (cover pictured to the left) focuses on explaining how measurement can be done simply and effectively for campaigns.
The publication costs 25 GDP but you can download the first section for free (pdf)>>
Presenting campaign evaluation at Berlin Summit
I’m happy to announce that I will be speaking at the forthcoming European Summit on Measurement, June 10-12 in Berlin, Germany. My presentation will be on an evaluation I have been doing on the global communications campaign on the 60th anniversary of the Universal Declaration of Human Rights (that’s the campaign logo on this post). I present jointly with Victor Fernandez of UN Human Rights.
Also presenting at the Berlin Summit, will be my follow blog author, Richard Gaunt. The Summit has launched a survey on the state of communications and PR measurement – which Richard will be presenting at the Summit. If you are a communications professional, please complete the survey here>>
Looking forward to seeing some fellow evaluators in Berlin!
Glenn
Six outcome categories for advocacy campaigns
As I mentioned in an earlier post, an interesting guide on measuring advocacy and policy (pdf) has been published by the Annie E. Casey Foundation. What I found interesting in this guide was that the authors have determined six outcome categories for advocacy campaigns. Studying campaigns, they identified what advocacy campaigns aim to achieve (“outcomes”) and broke it down into six categories:
1) Shift in social norms: e.g. knowledge, values, behaviour of society
2) Strengthened organisational capacity: e.g. skill set and structure of coalitions that carry out advocacy work.
3) Strengthened alliances: e.g. structural changes in community and insitutional relationships and alliances.
4) Strengthened base of support: e.g. grassroots, leadership and institutional support for policy changes.
5) Improved policies: e.g. policy development, demonstration of support, adoption, funding and implementation.
6) Changes in impact: e.g. ultimate change in social and physical lives and conditions.
Read more about these outcomes (go to page 17) in the guide (pdf).
Guide to measuring advocacy and policy
Here is an very useful guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation. The guide is accompanied by a handbook of data collection tools (pdf). They are both very interesting documents if you are interested in evaluating advocacy campaigns.
Glenn
Evaluating Public Information and Advocacy Campaigns
Last week, I was at the European Evaluation Society Biennial Conference in Lisbon, Portugal and presented a paper on “Evaluating public information and advocacy campaigns”.
Here is a summary of the paper:
Increasingly non-governmental organisations and international organisations use public information and advocacy campaigns to support their goals. Existing methodologies are rarely applied to evaluate campaigns. However, meaningful evaluation of campaigns is possible by taking into account the specific nature of campaigns while meeting minimum requirements of evaluation. This paper discusses “lessons learnt” in evaluating campaigns and particular challenges faced in assessing international campaigns. Although a standard methodology is yet to emerge, this paper describes the desired outcomes that many campaigns share and the appropriate evaluation methods that have been successfully used.
Read the full paper here (pdf)>>
Glenn
Guidelines for Evaluating Nonprofit Communications Efforts
For those interested in the area of campaign/advocacy evaluation, here are some very good guidelines on evaluating nonprofit communications efforts (pdf) from the Communications Consortium Media Center. The guidelines focus on “lessons learnt” in the evaluation of campaigns and sets out some key concepts to consider.
Glenn
Conference – building for the future: evaluation in governance, development & progress
The European Evaluation Society has it’s biennial conference (full title above) scheduled for 1-3 October 2008 in Lisbon, Portugal. The conference kicks off with a series of hands-on workshops, that I attended at the last conference and found very useful. Read more on the conference website.
I’ll be presenting a paper at the conference on “Evaluating Public Information and Advocacy Campaigns”, so if you are going, please drop me a line…
Glenn
Evaluating advocacy campaigns – No. 2
I’ve written previously about work that others and myself have done on evaluating communication and advocacy campaigns, particulary concerning campaigns that aim for both changes in individual behaviour and government/private sector policies.
In this area, here is an interesting article from the Journal of Multidisciplinary Evaluation, “Advocacy Impact Evaluation” (pdf) by Michael Q. Patton. The article explains how an evaluation was undertaken to evaluate the impact of an advocacy campaign to influence a decision of the US Supreme Court.
What I find interesting is how the evaluation was done – what is called the “General Elimination Method”.
This is where there is an effect (the Supreme Court decision) and an intervention (the advocacy campaign) and they search for connections between the two. They tried to eliminate alternative or rival explanations until the most compelling explanation remained. They did this through interviews, analysis of news, documents and the Court’s decision. The article explains all of this and makes for interesting reading, you can read the article here (pdf).
Glenn
Changing behaviour – takes a long time?
In an interesting post by Tom Watson on the Dummyspit blog, he writes about the difficulty of changing behaviour through PR campaigns – in this case to encourage better eating habits in UK schools (staying away from hamburgers for example). He also notes the rather depressing statistic that PR campaigns normally have only a 0.04% success rate of changing behaviour! In setting campaign objectives, I encourage organisations to be realistic about the targets they set – for example, a modest 2 – 10% behaviour change for public health campaigns is cited as being a realistic target (Rice & Paisley 1982).
But before you abandon your PR activities in despair, we have to be clear about what we define as “behaviour change”. Changing eating habits is a long-term behaviour change. PR campaigns often focus on trying to change long term behaviour and don’t consider setting goals in terms of immediate responses – what can people be expected to do when reached by the campaign? In PR terminology this is called “outtakes” (different from long-term behaviour which would be “outcomes”).
Following are some practical examples of “outtakes” or immediate responses (which are short term behaviours) that can be measured. I believe that the percentage of change desired amongst a target audience can can be set at higher than 10% for many of these:
– % of people that sign a petition
– % of people that refer a web page to a friend
– % people that go online to participate in an online discussion
– % of organisations/individuals that publicly support a campaign
– % of people who report undertaking a new initiative as a result of a campaign
Glenn
intelligent measurement 