Posts filed under ‘Campaign evaluation’
Advocacy campaigns and policy influence
Influencing policy is often an aim of many advocacy campaigns – the notion of trying to bring about change in the policy of governments, the private sector or even international organisations (e.g. UN). Here are two interesting publications in this area:
“Pathways for change: 6 Theories about How Policy Change Happens (pdf)” of the US-based Organization Research Services describes different theories as to how policy change can occur – interesting reading for those trying to influence policy.
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence – i.e. how you can evaluate your efforts to influence policy.
Both publications are worth a read if you are interested in policy influence and advocacy campaigning.
Online campaigning handbook
Here is a newish (well I just discovered it..) online campaigning handbook (pdf) from Publiczone.
Point 10 of the handbook “Keeping track of what you are doing” focuses on monitoring and evaluating online campaigns. Here is an extract of what they recommend:
Effective monitoring and evaluation can make the difference between an average and an amazing campaign. Monitor and evaluate as you go along and you’ll keep finding new opportunities to optimise your campaigning…The trick is to design your evaluation before you start, paying close attention to how you are going to collect data. Too often, charities leave evaluation to the end, only to discover they can only form a patchy picture of their campaign due to an absence of data.
E-Newsletter on advocacy evaluation
For those interested in advocacy evaluation, here is an interesting e-newsletter “Advocacy Evaluation Update” that is published some four times per year by the Innovation Network and the Center for Evaluation Innovation.
You can also read the latest edition (pdf) here>>
Researching campaign messages – wrinkles or cancer?
Here is a fascinating study – and story – about message effectiveness of campaigns.
For years, campaigns against excessive use of sunbeds (pictured above) – which are used to tan your skin – have focused their messages on the fact that excessive use can cause skin cancer.
Now a study from Northwestern University (USA) has found that the most persuasive message for young women is not the risk of cancer – but the risk of getting wrinkles.
It is an interesting example of where testing messages on the intended audiences can really help to determine the most effective message for a given group.
Challenges of evaluating advocacy campaigns
For those interested in evaluating advocacy campaign, here is an interesting resource:
The publication, from the California Endowment focuses on evaluating advocacy for policy changes from the perspective of foundations and NGOs. They also highlight some challenges to advocacy evaluation, including:
- Complexity
- Role of External Forces
- Time Frame
- Shifting Strategies and Milestones
- Attribution
- Limitations on Lobbying
- Grantee Engagement
What they say about “attribution”, that is to what extent can we attribute the change seen in policy or other developments to an individual campaign, is very much my experience also:
“It is also important to note that most policy work involves multiple players often working in coalitions and, in fact, requires multiple players “hitting” numerous leverage points. In this complex system, it is difficult to sort out the distinct effect of any individual player or any single activity…Even when an evaluator has the luxury of interviewing all key stakeholders involved in a policy development, there’s rarely, if ever, clear agreement on why something turned out the way it did. That can be frustrating to funders who want to pinpoint their influence.”
Read more about these challenges in the publication (pdf)>>
Communications and behaviour change
Here is a fascinating study produced by UK government’s Central Office of Information that summarises what influences people’s behaviour and the implications for communicators seeking to influence it.
Some of the key implications for communication programmes and campaigns that come out of the study include:
- Communications should not be viewed in isolation
- Developing a practical behavioural model can help make communications more effective at influencing behaviour
- Paid-for media opportunities (which traditionally account for the biggest part of the government communications budget) are not always the most trusted sources
- Understanding behaviour and its influences will enable us to harness the most efficient and effective communications channels.
- Understanding behaviour will support more robust and meaningful evaluation.
Evaluating advocacy and policy change
This 32-page issue of The Evaluation Exchange (pdf) from the Havard Family Research Project describes new developments in evaluating advocacy and policy change – excellent points on the challenges and best practices for evaluating advocacy campaigns.
Standardisation of PR Evaluation Metrics?
The UK government’s Central Office of Information (yes, I know it’s rather Orwellian sounding…) has produced a set of metrics for measuring PR campaigns: “Standardisation of PR Evaluation Metrics” (pdf).
Frankly, they are disappointing. The title is deceiving, it should be called “Standardisation of media monitoring metrics”. As that all the document covers – the superficial stuff – “reach”, “Favourability of coverage”, etc. These are “outputs” of PR activities.
But what about metrics for measuring “outcomes”? These don’t get a mention. Well there is an admission but you have to dig deep, they do say:
“It is worth bearing in mind that these standardised core metrics for media evaluation are only one component of any campaign evaluation. It is crucial to agree specific key performance indicators (KPIs) at the outset of a campaign.”
So they admit it, these are metrics for media coverage only. And no guidance is given on these KPIs (that are typically “outcome” level). If you are interested in learning more about metrics for “outcomes”, I’d recommend you start with the excellent guide from the Institute of PR: “Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities” (pdf).
New resources on advocacy evaluation
Here are some new resources on evaluating advocacy campaigns:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
For further resources and information on advocacy and campaign evaluation, please consult past posts on these subjects.
Evaluating Strategic Communication for Avian Influenza/Pandemic Influenza
As the world is focused on the current flu pandemics, we have seen many efforts to communicate prevention and treatment approaches.
And what about how to measure the results of such communication efforts? Here is an interesting set of guidelines from UNICEF on this issue:
Although it’s a technical document, it provides interesting insight into sampling and interviewing techniques for evaluating communication campaigns.
intelligent measurement 