Posts filed under ‘Evaluation use’
Getting the final evaluation report right / write
For many evaluation projects, an important “deliverable” is the final evaluation report, which contains the findings, conclusions and recommendations of the evaluation. Having been through many evaluations as part of a team or as an individual, I am surprised at how often this important step gets neglected or simply messed up. Following are a couple of recommendations on putting together a final evaluation report:
- Link the findings to the original evaluation questions: Not my own idea, but something I’ve seen others do well – structure the findings of the evaluation around the original questions from the brief that defined the evaluation. In this way, people reading the report can make the connection between the questions asked and what was found out.
- Summarise the key findings in one diagram or table: Aside from reading the executive summary, people often appreciate grasping the key results in one view. Without vulgarising the findings, I find it is useful to sumarise the key findings visually. You can see an example of this idea (called a “snapshot”) on page five of this evaluation report (pdf).
- Separate the recommendations from the findings: Often you see recommendations spread throughout the main body of the report. I find it confusing and believe it is easier to go through recommendations when they are found after the findings (while still making clear reference to the findings).
- Make the executive summary a summary: An executive summary should be just that – a summary. I’m surprised at how many reports actually include new information in their executive summaries that are not found elsewhere in the reports. I recommend summarising the main findings and touching on the recommendations if space allows.
- Include all the details for the really interested and pedantic: There will be a small number of your readers that will love to look further into the details – read all the 1000s of responses to the open questions, study the way the sample was selected, etc. For these readers, I recommend including these details of the evaluation as annexes. These details, such as the survey questions, interview guidelines, description of methodology, further analysis of demographics, existing research consulted, etc. will only strengthen your report and answer some questions for a select group of readers.
Related to this topic, I’ve also written previously about how to ensure that your results are used and how to present monitoring and evaluation results.
And if you want to read further, here are some very comprehensive guidelines from the World Bank on Presenting Results (pdf).
Glenn
Six factor to ensure that your evaluation results are used
As I wrote in a previous post, evaluation can be quite frustrating when all your effort and work doesn’t actually lead to any significant change in future practices. Why are evaluations not used? A new report “The Utilisation of Evaluations” (pdf) from ALNAP throws some light on this subject. Although focusing on the humanitarian sector, the report has some points that apply to all types of evaluations. I found interesting the six quality factors the author identifies that contribute to the findings of an evaluation being utilised, notably:
- Designing carefully the purpose and approach of the evaluation
- Managing quality participation of all stakeholders throughout the evaluation
- Allowing enough time to have all relevant staff and stakeholders involved
- Ensuring that the evidence is credible and the report is easy to read with clear, precise recommendations with who is responsible for what and when
- Putting in place follow-up plans at the outset
- Ensuring that the evaluator(s) are credible, balanced and constructive – wholesale negativity is never welcomed
Going through these six factors I can see where I’ve faced obstacles in past evaluations, notably points 2 and 5. I find managing stakeholder involvement is often difficult and so is setting out follow-up plans – it often comes as an after-thought. Certainly some factor to consider for all evaluators…
Read the full report (pdf) here >>
Glenn
Presenting monitoring & evaluation results
The more I work in the M&E field, the more I see the importance of presenting results in a consumable way. If you are leading an evaluation project, there is nothing more frustrating than finishing your project and finding the comprehensive report you wrote gathering dust on a manager’s desk.
But that’s what I have learnt, the comprehensive report will perhaps only be read by one or two people of the commissioning team – but the powerpoint summarising the report will be widely distributed and viewed by many. We may think this is a “dumbing-down” of the work undertaken but it is a reality of how our work is consumed. Here are some points on presenting results that I find useful:
- Think carefully about the data and findings you want to present. We can often be overwhelmed by data (from survey results for example). If in doubt, put data you consider important but not essential in report annexes.
- Make the evaluation report attractive and easy to ready – facilitate this by summarising the main points and creating a brief presentation.
- Organise an event such as a staff or team meeting to discuss the results – this could have more impact than the written document.
- Through blogs and wikis, use the evaluation results to generate more discussion and interest in the given subject. A good example is the blog created to present the results of the 2006 Euroblog survey.
Jim Macnamara in a recent article (pdf) touches on this subject on how presenting results with a “two-tier” approach is useful – that is, presenting to top management only key data and information while fully digesting all data at the corporate communications level.
Glenn
Cartoon from toothpaste for dinner>>
intelligent measurement 