Posts filed under ‘Advocacy evaluation’
advocacy evaluation: influencing climate change policy
Often I don’t get to share the findings of the evaluations I undertake, but in this case of an advocacy evaluation, an area that I’ve written about before, the findings are public and can be shared.
I was part of a team that evaluated phase 1 of an advocacy/research project – the Africa Climate Change Resilience Alliance (ACCRA). ACCRA aims to increase governments’ and development actors’ use of evidence in designing and implementing interventions that increase communities’ capacity to adapt to climate hazards, variability and change. Advocacy plays a large role in trying to influence governments and development actors in this project. You can read more in the Executive_Summary (pdf) of the evaluation findings.
The evaluation also produced 5 case studies highlighting successesful advocacy strategies:
- Capacity building and district planning
- Secondment to a government ministry
- Reaching out to government and civil society in Uganda
- Disaster risk profiling in Ethiopia
- Exchanging views and know-how between ACCRA countries
The case studies can be viewed on the ACCRA Eldis community blog (n.b. you have to join the Eldis community to view the case studies, it’s free of charge).
To disseminate the evaluation findings widely we also produced a multimedia clip, as featured below.
Advocacy evaluation – top resources
Today I spoke to the students of the Executive Certificate of Advocacy in International Affairs at the Graduate Institute of Geneva on advocacy evaluation. I promised the students to list the top resources I’d recommend on advocacy evaluation, here they are:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
Guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence.
“Advocacy Impact Evaluation” (pdf) by Michael Q. Patton – an interesting case study on influencing the US Supreme Court.
“Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating advocacy campaigns ranging from gun safety to emmissions (ozone) reduction.
A pragmatic guide to monitoring and evaluating research communications using digital tools
The approach taken relates online measurement tools to four levels of assessing influence of communications on policy (an aim of research communications):
- Management, outputs, uptake, outcomes and impact.
The last level, outcomes and impact is of course the hardest to measure with digital tools. But I think if you have access to your target audiences, this can be done through in-depth interviews or more simply through email surveys to ask how they have used the research products – which can give then provide an indication of the role they have taken in influencing policy.
EU Manual: Evaluating Legislation and Non-Spending Interventions in the Area of Information Society and Media
A very interesting manual published by the European Union:
Despite the wordy title, the manual is really about how to evaluate the effects of legislation and initiatives taken by governments (in this case the regional body – EU).
The toolbox at page 72 is well worth a look.
The Elusive Craft of Evaluating Advocacy
Here is an interesting article “The Elusive Craft of Evaluating Advocacy” (pdf) that examines some of the challenges of undertaking advocacy evaluation, mostly from the US perspective.
The authors consider advocacy evaluation more of a “craft” than an exact science:
“The real art of advocacy evaluation, which is beyond the reach of quantitative methods, is assessing influence..Advocacy evaluation is a craft—an exercise in trained judgment—one in which tacit knowledge, skill, and networks are more useful than the application of an all-purpose methodology.”
Advocacy impact Assessment Guidelines
Here is an interesting fact sheet from CABI.org – “Advocacy Impact Assessment Guidelines” (pdf).
The fact sheet provides a very good summary of evaluating advocacy actions – the “how” and “what” to evaluate. It also highlights some key points to keep in mind, summarised here:
- Different stakeholders will have different views on what success is;
- If you cannot prove impact, be satisfied with a critically informed assessment of change;
- Include subjective criteria, (i.e. what successes people feel have taken place but cannot substantiate with evidence);
- Break down your advocacy intervention into manageable components;
- Be practical, yet flexible. The external environment in which your advocacy takes place will be changing all the time;
- Monitor changes in your strategy itself;
- Collaborative advocacy means that individual contributions cannot be separated from the success of the whole effort;
- Share evaluation results with a wide range of people to show the disbelievers that advocacy can work and to motivate those who have been involved.
View the fact sheet here (pdf) >>
Ten takeaways on evaluating advocacy and policy change
The Harvard Family Research Project produce some excellent material on advocacy evaluation.
From their newsletter(pdf), here are ten takeways on evaluating advocacy and policy change:
1.Advocacy evaluation has become a burgeoning field.
2. Advocacy evaluation is particularly challenging when approached with a traditional program evaluation mindset.
3. The goals of advocacy and policy change efforts—that is, whether a policy or appropriation was achieved—typically are easy to measure.
4. Many funders’ interest in advocacy evaluation is driven by a desire to help advocates continuously improve their work, rather than to prove that advocacy is a worthy investment.
5. Advocates must often become their own evaluators. Because of their organizational size and available resources, evaluation for many advocates requires internal monitoring and tracking of key measures rather than external evaluation.
6. External evaluators can play critical roles.
7. Context is important.
8. Theories of change and logic models that help drive advocacy evaluation should be grounded in theories about the policy process.
9. Measures must mean something.
10. Evaluation creativity is important.
Advocacy campaigns and policy influence
Influencing policy is often an aim of many advocacy campaigns – the notion of trying to bring about change in the policy of governments, the private sector or even international organisations (e.g. UN). Here are two interesting publications in this area:
“Pathways for change: 6 Theories about How Policy Change Happens (pdf)” of the US-based Organization Research Services describes different theories as to how policy change can occur – interesting reading for those trying to influence policy.
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence – i.e. how you can evaluate your efforts to influence policy.
Both publications are worth a read if you are interested in policy influence and advocacy campaigning.
E-Newsletter on advocacy evaluation
For those interested in advocacy evaluation, here is an interesting e-newsletter “Advocacy Evaluation Update” that is published some four times per year by the Innovation Network and the Center for Evaluation Innovation.
You can also read the latest edition (pdf) here>>
Evaluating advocacy and policy change
This 32-page issue of The Evaluation Exchange (pdf) from the Havard Family Research Project describes new developments in evaluating advocacy and policy change – excellent points on the challenges and best practices for evaluating advocacy campaigns.
intelligent measurement 