New publication: Learning about Measuring Advocacy and Policy Change: Are Baselines always Feasible and Desirable?
IDS have produced an new paper “ Learning about Measuring Advocacy and Policy Change: Are Baselines always Feasible and Desirable?” (pdf). Here is a summary from the author:
This paper captures some recent challenges that emerged from establishing a baseline for an empowerment and accountability fund. It is widely accepted that producing a baseline is logical and largely uncontested – with the recent increased investment in baselines being largely something to be welcomed. This paper is therefore not a challenge to convention, but rather a note of caution: where adaptive programming is necessary, and there are multiple pathways to success, then the ‘baseline endline’ survey tradition has its limitations. This is particularly so for interventions which seek to alter complex political-economic dynamics, such as between citizens and those in power.
The paper raises some very valid points about the challenges of establishing baselines, particularly for advocacy/policy change projects – one which I’ve also experienced in that with advocacy we are never rarely starting from “zero” – organisations could have been working on a given issue for some time when this given project came along.
1) Integrating Communication in Evaluation
The Better Evaluation blog has published a series of posts focusing on the use of video in evaluation:
I’ve been reviewing what handbooks and guides are available on communication evaluation – so far I’ve located five, here they are (all links to PDFs):
Communication Network (2008) Are we there yet? A Communications Evaluation Guide
DFID (2005). Monitoring and Evaluating Information and Communication for Development (ICD) Programmes – Guidelines. Department for International Development, London.
Slighty more specialised, but still interesting:
The Coalition for Public Relations Research Standards has just released their interim metrics (pdf) on PR research and measurement. They’ve split it by traditional media, social media, communications lifestyle and Return on Investment (ROI). For each metric, there is also more information available. I’m yet to go through all the information, but it seems like a comprehensive list – even if not all will agree on the different definitions, etc. Of interest, four major US corporations – GE, GM, McDonald’s USA and Southwest Airlines – reportedly have already adopted these metrics.
For those who use graphs in evaluation reports and other documents, here is an excellent presentation from Ann K. Emery of the Innovation Network – well worth a look!
An often-overlooked step in evaluation is ensuring that findings are communicated, understood and acted upon. Equally important, however,is what, how and when we communicate with different stakeholders throughout the evaluation process. Communicating effectively implies using different means,messages and methods to reach different groups with very different needs and expectations.
2) Addressing complexity in evaluation – June 5 and 6, 2014
Increasingly evaluations have to address programs, projects and policies with complex aspects. The activities and objectives of these interventions are fundamentally dynamic and emergent in response to needs and opportunities, and they often involve multiple organisations with emergent and unpredictable roles. These characteristics present challenges to traditional evaluation approaches.
The workshop will examine the particular challenges that complexity presents and explore practical strategies for evaluation, including developmental evaluation, use of non-linear logic models, and emergent evaluation design. The workshop will include case studies of successful and unsuccessful attempts to address complexity in evaluation. It will also provide opportunities to analyse participants’ own examples in terms of identifying the particular challenges that complexity presents and how the different strategies might be applied.
The workshop will be facilitated by Professor Patricia Rogers, Royal Melbourne Institute of Technology, Australia.
Both courses will will be held in the Federal Office of Personnel, Bern, Switzerland. Please note that there are no scholarships or travel funds available for these courses.