Posts filed under ‘Advocacy evaluation’
New resources on advocacy evaluation
Here are some new resources on evaluating advocacy campaigns:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
For further resources and information on advocacy and campaign evaluation, please consult past posts on these subjects.
Six outcome categories for advocacy campaigns
As I mentioned in an earlier post, an interesting guide on measuring advocacy and policy (pdf) has been published by the Annie E. Casey Foundation. What I found interesting in this guide was that the authors have determined six outcome categories for advocacy campaigns. Studying campaigns, they identified what advocacy campaigns aim to achieve (“outcomes”) and broke it down into six categories:
1) Shift in social norms: e.g. knowledge, values, behaviour of society
2) Strengthened organisational capacity: e.g. skill set and structure of coalitions that carry out advocacy work.
3) Strengthened alliances: e.g. structural changes in community and insitutional relationships and alliances.
4) Strengthened base of support: e.g. grassroots, leadership and institutional support for policy changes.
5) Improved policies: e.g. policy development, demonstration of support, adoption, funding and implementation.
6) Changes in impact: e.g. ultimate change in social and physical lives and conditions.
Read more about these outcomes (go to page 17) in the guide (pdf).
Guide to measuring advocacy and policy
Here is an very useful guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation. The guide is accompanied by a handbook of data collection tools (pdf). They are both very interesting documents if you are interested in evaluating advocacy campaigns.
Glenn
Evaluating Public Information and Advocacy Campaigns
Last week, I was at the European Evaluation Society Biennial Conference in Lisbon, Portugal and presented a paper on “Evaluating public information and advocacy campaigns”.
Here is a summary of the paper:
Increasingly non-governmental organisations and international organisations use public information and advocacy campaigns to support their goals. Existing methodologies are rarely applied to evaluate campaigns. However, meaningful evaluation of campaigns is possible by taking into account the specific nature of campaigns while meeting minimum requirements of evaluation. This paper discusses “lessons learnt” in evaluating campaigns and particular challenges faced in assessing international campaigns. Although a standard methodology is yet to emerge, this paper describes the desired outcomes that many campaigns share and the appropriate evaluation methods that have been successfully used.
Read the full paper here (pdf)>>
Glenn
Guidelines for Evaluating Nonprofit Communications Efforts
For those interested in the area of campaign/advocacy evaluation, here are some very good guidelines on evaluating nonprofit communications efforts (pdf) from the Communications Consortium Media Center. The guidelines focus on “lessons learnt” in the evaluation of campaigns and sets out some key concepts to consider.
Glenn
Evaluating advocacy campaigns – No. 2
I’ve written previously about work that others and myself have done on evaluating communication and advocacy campaigns, particulary concerning campaigns that aim for both changes in individual behaviour and government/private sector policies.
In this area, here is an interesting article from the Journal of Multidisciplinary Evaluation, “Advocacy Impact Evaluation” (pdf) by Michael Q. Patton. The article explains how an evaluation was undertaken to evaluate the impact of an advocacy campaign to influence a decision of the US Supreme Court.
What I find interesting is how the evaluation was done – what is called the “General Elimination Method”.
This is where there is an effect (the Supreme Court decision) and an intervention (the advocacy campaign) and they search for connections between the two. They tried to eliminate alternative or rival explanations until the most compelling explanation remained. They did this through interviews, analysis of news, documents and the Court’s decision. The article explains all of this and makes for interesting reading, you can read the article here (pdf).
Glenn
Evaluating Advocacy Campaigns
I’ve written previously about work that others and myself have done on evaluating communication campaigns, particulary concerning campaigns that aim for both changes in individual behaviour and government/private sector policies. In this same direction, a post from the Mission Measurement blog caught my eye on evaluating advocacy campaigns. They make the very relevant point that although evaluating the impact of advocacy campaigns is difficult – trying to isolate the precise influence on changes being observed – what certainly can be measured is the progress towards the desired change.
They go on to provide some further insight into this issue, by looking at various measurements undertaken, such as:
- Number of contacts established
- Intermediate changes to knowledge/attitudes
- Measuring progress of change on a continuum
- Bellweather ratings
In the same vein, what I recommend to organisations is to set clear objectives to start with in terms of what is precisely expected from advocacy/campaigning and establish relatively simple “tracking mechanisms” to follow “progress” on an issue – on a policy level (e..g. number of governments that publicly commit to a given issue) or at an individual level (e.g number of people who pledge to undertake a given action). Often this information is “known” within an organisation but is not centralised or analysed – making any conclusion on a campaign’s impact difficult.
Glenn
Evaluating Communication Campaigns
As the whole development and humanitarian sector focus more on accountability and performance, there has been a push for more evaluation of communication activities of this sector.
Most methodology and tools can be adapted from those used in the private sector. However, many communication campaigns of NGOs and international organisations often have dual outcomes they wish to achieve – individual behaviour change (e.g. persuade individuals to adopt a more healthier lifestyle) and policy change (e.g. push governments to change policy on food labeling).
An excellent study “Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating campaigns ranging from gun safety to emmissions (ozone) reduction.
If you are interested to read more about standards and practices of evaluation in the development and humanitarian sector, a good starting point is The Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP). This interagency forum focuses on evaluation and accountability issues in this sector.
Glenn
intelligent measurement 