New paper: Implementing Development Evaluations under Severe Resource Constraints
A very interesting paper from the Center for Development Impact on implementing evaluations with limited resources, here is a summary:
Most agency evaluations are very short both on resources and in duration, with no proper opportunity to assess impact in a valid manner. The methodology for these evaluations is based on interviews, a review of available programme literature and possibly a quick visit to one (often unrepresentative and usually successful) programme site. This means that the results of the evaluations are heavily dependent on the experience and judgement of the evaluator, the opinions received, and level of support from the commissioner. This CDI Practice Paper by Richard Longhurst reviews how to make the best of such a situation, drawing on lessons learned from techniques of better resourced evaluations and other techniques that have been used. A simple framework can relate the type of evaluation to the resources available and enable better planning and use of evaluations across an organisation.
Advocacy evaluation: mini tool kit
Really great concise advocacy evaluation mini-tool kit (pdf) – some very useful ideas on activity tracking and measuring support for a given issue.
Evaluation reporting: A guide
From the US CDC, here is an interesting guide: Evaluation reporting: A guide to help ensure use of evaluation findings (pdf).
It contains many tips and advice on how to craft your evaluation report to make it more *usable*…
more resources on advocacy evaluation
Below are some resources on advocacy evaluation that I’ve come across recently (they may not be *new* but they are to me!)
Advocacy evaluation: Review and Opportunities (pdf): Justin Whelan (2008) reviews the emerging literature in this field, noting the points of convergence and divergence and suggesting some limitations of the frameworks and opportunities for effective evaluation that meets the needs of interest groups.
Advocacy evaluation: What it is and where to find out more about it (pdf): Various authors from the health field define advocacy evaluation and provide a list of resources.
Unique methods in advocacy evaluation (pdf): From Julia Coffman and Ehren Reed, an interesting article on some evaluation methods that are considered unique to advocacy evaluation.
Still not had enough? Read my past posts on advocacy evaluation>>
video reporting: realtime evaluation
I’ve blogged before about the use of video in presenting evaluation findings. Here is a good example of how video has been used to present the process and some key results of a real-time evaluation of the 2010 floods in Pakistan. The video tells the story well even if it is missing some background information for the viewer (what, why and for whom was the evaluation).
New article: Evaluation of IO/INGO communication activities
I’ve just had an article published in the journal PR Review. It’s the first article of my ongoing PhD on communication evaluation in intergovernmental organizations and NGOs. Below is the Abstract or if you are really keen you can download the full article below.
Abstract
Evaluation of international and non-governmental organizations’ communication activities: A 15 year systematic review
The purpose of this paper is to understand how intergovernmental organizations and international non-governmental organizations have evaluated their communication activities and adhered to principles of evaluation methodology from 1995–2010 based on a systematic review of available evaluation reports (N = 46) and guidelines (N = 9). Most evaluations were compliant with principle 1 (defining communication objectives), principle 2 (combining evaluation methods), principle 4 (focusing on outcomes) and principle 5 (evaluating for continued improvement). Compliance was least with principle 3 (using a rigorous design) and principle 6 (linking to organizational goals). Evaluation was found not to be integrated, adopted widely or rigorously in these organizations.
Presenting evaluation findings – Using Prezi
I’ve written before about the challenges of presenting evaluation findings – and the need to find creative ways to present findings. No doubt many readers would have already come across Prezi, a new presentation software that you see used often these days. I’ve recently worked on a research project for the Joint Standards Initiative where we presented our research results in a Prezi as you can see displayed below. It is allows the results to presented in visually appealing way and I believe communicates well the main points.
[gigya src=”http://prezi.com/bin/preziloader.swf” allowfullscreen=”true” allowscriptaccess=”always” width=”550″ height=”400″ bgcolor=”#ffffff” flashvars=”prezi_id=8m7rw0lpp9ew&lock_to_path=0&color=ffffff&autoplay=no&autohide_ctrls=0″ ]
This Prezi was produced by Peter from YourPrezi.
A guide to monitoring government policies
We often focus on the evaluation of advocacy that aims to influence government policies – but another important aspect is to monitor government policy as it develops (or not).
Here is a useful guide from CAFOD, Trocaire and Christian Aid on this subject: Monitoring government policies: A toolkit for civil society organisations in Africa (pdf).
It explains very well how policy is developed and when and how it can be monitored – very informative.
Data visualization for evaluation
If you are interested in how evaluation results are presented as I am, then you might be interested in this fact sheet (pdf) from IDRC on data visualization for communicating research and evaluation findings. It contains some fascinating information on how data visualization can be used (and misused). As can be seen in the chart below it shows that evaluation reports are on the low end in terms of their usage of data visualization.
The PR Agency Of The Future – measurement and data!
Paul Holmes of the Holmes Report has written a very interesting article on what he sees as key to the future of the PR agency.
For those interested in PR measurement, what is reassuring is the focus he puts on the need for the better use of data and measurement by agencies. I’m always surprised to see how little PR agencies do in measurement – so any more uptake of evaluation and measurement would be welcome.
Here is a summary of some key points:
- Big data at the center: Sufficient evidence suggests data and analytics can have a powerful effect on communications. There has been an incremental increase in the use of data to drive PR efforts, but the progression is minimal.
- Insight to drive meaningful creativity: Strong data will lead to better insights, giving way to creative PR ideas that effectively solve real world problems. Don’t assume your experience is enough to make a good campaign – use data.
- Understanding the human brain: To better understand how to change behaviors and attitudes, PR pros should read and listen to neuroscientists like David Eagleman. After all, PR is a social science.
- Recruiting differently: Practitioners who understand and even love data exist, but firms need to recruit a broader, more digestive range of people to find them. Seemingly unrelated disciplines should not be ruled out.
- Make it matter: To ensure communications efforts pay off in business terns, every campaign, every stakeholder group, and every advance in how we apply data and science can and should be measured.
intelligent measurement 