Posts filed under ‘Campaign evaluation’
Evaluation report – Oxfam’s GROW campaign
For readers interested in campaign evaluation, Oxfam has just published a mid-point external evaluation report (pdf) of their GROW campaign – of which I was part of the evaluation team.
Often organisations will not make available publically their campaign evaluations – but Oxfam has a progressive policy on this so I’m happy to be able to share the report will all interested…
The GROW campaign set out in 2011 to tackle food justice and build a better food system. Challenging to evaluate, the GROW campaign is broad and diverse, operating at national, regional and international levels, across 4 thematic areas – land, investment in small-scale agriculture, climate change and food price volatility.
In our evaluation report we look at the initial Theory of Change and endeavour to track the changes seen over the first two years and the possible intervening factors, positive and negative, using a variety of methods including five case studies (found at the end of the report).
As the campaign had a broad set of activities at a range of levels, the challenge for the evaluation team was to capture all significant changes seen to date and draw out learnings for the future.
Oxfam has also produced a summary infographic that you can view below.
View the executive summary (pdf)>>
The executive summary is also available in French (pdf) and Spanish (pdf) – and you can also read Oxfam’s management response to the evaluation (pdf).
Advocacy M&E: examples from Global Witness, Save, Amnesty and CDKN
Below are some useful examples – presentation posters – of how leading NGOs are tackling advocacy monitoring and evaluation:
Developing a Planning, Monitoring, Evaluation and Learning (PMEL) System for Global Witness. Jenny Ross (Consultant and INTRAC Associate) and Gavin Hayman (Global Witness)
http://www.intrac.org/data/files/NGO_Forum_2013/Jenny_Ross_Global_Witness_-_final.pdf
Tracking advocacy efforts: Save the Children’s Advocacy MonitoringTool. Save the Children
http://www.intrac.org/data/files/NGO_Forum_2013/Save_the_Children_-_final.pdf
Supporting international climate negotiators: a monitoring and evaluation framework. Climate and Development Knowledge Network (CDKN)
http://www.intrac.org/data/files/Afternoon/CDKN_-_final.pdf
Using ‘theory of change’ approach to assess human rights advocacy. Amnesty International
http://www.intrac.org/data/files/Afternoon/3Amnesty_International_-_final.pdf
These posters are from an INTRAC training workshop – thanks to Jenny Ross for highlighting them.
Visualizing Information for Advocacy
Just came across this interesting guide(pdf) to using visuals for advocacy from the Tactical Technical Collective – here is an explanation from the authors:
Visualising Information for Advocacy(pdf): An Introduction to Information Design is a manual aimed at helping NGOs and advocates strengthen their campaigns and projects through communicating vital information with greater impact. This project aims to raise awareness, introduce concepts, and promote good practice in information design – a powerful tool for advocacy, outreach, research, organisation and education. Through examples, the booklet demonstrates how to use innovative visual graphics to tell a complex and powerful story in a snapshot.
Evaluation of communication activities of international and non-governmental organisations: A 15 year systematic review
As part of my PhD studies, I have undertaken a systematic review of how international and non-governmental organisations are evaluating their communication activities. I’m presenting a summary of this today at the European Evaluation Society Conference in Helsinki, Finland. Below are the slides, hope you find them interesting.
New advocacy evaluation guide
Bond, the UK alliance of NGOs, has produced an interesting guide on advocacy evaluation:
Assessing effectiveness in influencing power holders (pdf)
The guide looks at the challenges of influencing power holders (usually done through activities grouped under the umbrella of “advocacy”) but comes to the conclusion that evaluation is feasible:
it is possible to tell a convincing story of an organisation’s contribution to change through their influencing and campaigning work by breaking down the steps of the process that led to change, and looking at how an organisation has created change at each step.
The guide also sets out these steps and provides examples of advocacy evaluation tools from NGOs including Oxfam, CARE, Transparency International amongst others.
Understanding public attitudes to aid and development
Here is a fascinating research paper on Understanding public attitudes to aid and development (pdf) from the UK-based ODI and IPPR.
Relevant to monitoring and evaluation, it recommends:
“Campaigns should do more to communicate how change can and does happen in developing countries, including the role aid can play in catalysing or facilitating this change. Process and progress stories about how development actually happens may be more effective communication tools than campaigns focused straightforwardly on either inputs (such as pounds spent) or outputs (such as children educated).”
This is a weakness of campaigning about development and aid, in that the steps towards change are not explained – the so-called “theory of change” – “if we do that – it will lead to that” – is a mystery – for the public and often those running the programmes have not always thought it through either…
Thanks to the Thoughtful Campaigner blog for bringing this to my attention.
advocacy evaluation: influencing climate change policy
Often I don’t get to share the findings of the evaluations I undertake, but in this case of an advocacy evaluation, an area that I’ve written about before, the findings are public and can be shared.
I was part of a team that evaluated phase 1 of an advocacy/research project – the Africa Climate Change Resilience Alliance (ACCRA). ACCRA aims to increase governments’ and development actors’ use of evidence in designing and implementing interventions that increase communities’ capacity to adapt to climate hazards, variability and change. Advocacy plays a large role in trying to influence governments and development actors in this project. You can read more in the Executive_Summary (pdf) of the evaluation findings.
The evaluation also produced 5 case studies highlighting successesful advocacy strategies:
- Capacity building and district planning
- Secondment to a government ministry
- Reaching out to government and civil society in Uganda
- Disaster risk profiling in Ethiopia
- Exchanging views and know-how between ACCRA countries
The case studies can be viewed on the ACCRA Eldis community blog (n.b. you have to join the Eldis community to view the case studies, it’s free of charge).
To disseminate the evaluation findings widely we also produced a multimedia clip, as featured below.
Advocacy evaluation – top resources
Today I spoke to the students of the Executive Certificate of Advocacy in International Affairs at the Graduate Institute of Geneva on advocacy evaluation. I promised the students to list the top resources I’d recommend on advocacy evaluation, here they are:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
Guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence.
“Advocacy Impact Evaluation” (pdf) by Michael Q. Patton – an interesting case study on influencing the US Supreme Court.
“Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating advocacy campaigns ranging from gun safety to emmissions (ozone) reduction.
Measuring the impact of NGO programmes
An ongoing debate focuses on how NGOs can measure the impact of their work. The International Initiative for Impact Evaluation (3ie) and Oxfam have recently produced a very interesting paper on this subject:
Using examples from campaigns and other programmes, the paper sets out the challenges and options in evaluating impact and proposes four options for improving impact evaluation:
1) partnering with research institutions to rigorously evaluate “strategic” interventions;
2) pursuing more evidence informed programming;
3) using what evaluation resources they do have more effectively;
4) making modest investments in additional impact evaluation capacity.
View the paper (pdf)
Ten takeaways on evaluating advocacy and policy change
The Harvard Family Research Project produce some excellent material on advocacy evaluation.
From their newsletter(pdf), here are ten takeways on evaluating advocacy and policy change:
1.Advocacy evaluation has become a burgeoning field.
2. Advocacy evaluation is particularly challenging when approached with a traditional program evaluation mindset.
3. The goals of advocacy and policy change efforts—that is, whether a policy or appropriation was achieved—typically are easy to measure.
4. Many funders’ interest in advocacy evaluation is driven by a desire to help advocates continuously improve their work, rather than to prove that advocacy is a worthy investment.
5. Advocates must often become their own evaluators. Because of their organizational size and available resources, evaluation for many advocates requires internal monitoring and tracking of key measures rather than external evaluation.
6. External evaluators can play critical roles.
7. Context is important.
8. Theories of change and logic models that help drive advocacy evaluation should be grounded in theories about the policy process.
9. Measures must mean something.
10. Evaluation creativity is important.
intelligent measurement 