Posts filed under ‘Evaluation methodology’
New e-learning course: Real-time evaluation and adaptive management
My friends at TRAASS have launched a new e-learning course on Real-time evaluation and adaptive management:
“What exactly is an RTE/AM approach and how can it help in unstable or conflict affected situations? Do M&E practitioners need to ditch their standard approaches in jumping on this latest bandwagon? What can you do if there is no counterfactual or dataset? This modular course covers these challenges and more.”
Tips for young / emerging evaluators
The Evaluation for Development blog from Zenda Ofir has been collating tips for young / emerging evaluators – that even experienced evaluators will find interesting. Here are some highlights:
From Zenda herself:
Top Tip 1. Open your mind. Read
Top Tip 2. Be mindful and explicit about what frames and shapes your evaluative judgments.
Top Tip 3. Be open to what constitutes “credible evidence”.
Top Tip 4. Focus a good part of your evaluative activities on “understanding”.
Top Tip 5. Be or become a systems thinker who can also deal with some complexity concepts.
Read more about these tips>>
From Juha Uitto:
Top Tip 1. Think beyond individual interventions and their objectives.
Top Tip 2. Understand, deal with and assess choices and trade-offs made or that should have been made.
Top Tip 3. Methods should not drive evaluations.
Top Tip 4. Think about our interconnected world, and implore others to do the same.
Read more about these tips>>
From Benita Williams:
Top Tip 1. The cruel tyranny of deadlines.
Top Tip 2. Paralysis from juggling competing priorities.
Top Tip 3. Annoyance when you are the messenger who gets shot at
Top Tip 4. Working with an evaluand that affects you emotionally
Top Tip 5. Feeling rejected if you do not land an assignment
Top Tip 6. Feeling demoralized when you work with people who do not understand evaluation
Top Tip 7. Feeling discouraged because of wasted blood sweat and tears
Top Tip 8. Feeling lazy if you try to maintain work-life balance when other consultants seem to work 24/7
Top Tip 9. Feeling overwhelmed by all of the skills and knowledge you should have
Read more about these tips>>
And from Michael Quinn Patton, just one tip:
Top tip 1: Steep yourself in the classics.
Read more about this tip>>
New ILO guidelines for evaluation
The International Labour Office (ILO) has released their new guidelines for evaluation (pdf). Although the guidelines are specific to the ILO, the guidelines contain many useful chapters for evaluators and evaluation commissioners in general, for example on the steps for planning and managing and evaluation and communicating the results. View the guidelines here (pdf)>>
Spotting dubious data
ACAPS has produced a great poster on “Spotting Dubious Data”. They make reference to humanitarian action but it applies across all sectors. Below is a simplified version of the poster.
I think particularly point 1. is often ignored when looking at data – WHY was this data collected…
New e-learning course: cutting edge M&E
A new e-learning course is available from TRAASS international ; Cutting-Edge M&E: A Guide for Practitioners. The course is taught by Colin Jacobs, a senior trainer with more than 25 years’ experience in international development. Colin’s recent roles include President at the UK Evaluation Society and Head, Governance and Civil Society at British Council.
This online course lays the ground for Monitoring and Evaluation (M&E) to make vital contributions to incentivising change and measure performance. The course considers challenges in current M&E practice, introduces a tool-box of evaluation techniques and shows where these can be best applied. Ways of promoting early participation and the engagement of key stakeholders are explored and a step-by-step action plan to improve practice of M&E is provided. Further information>>
Full disclosure; I also present an e-learning course for TRAASS International; Effective and creative evaluation report writing.
Event – The Future of Technology for Evaluation
A very interesting event is scheduled for February 20-21 2017 in London; the Future of technology for monitoring, evaluation, research and learning – MERL TECH; learn more about the event>>
New report: Evaluation Capacity and Practice in the US Nonprofit Sector
A very interesting report is just out from the Innovation Network on the Evaluation Capacity and Practice in the US Nonprofit Sector (pdf).
Here are some excerpts on resources and evaluation:
- 99% of organisations have someone responsible for evaluation
- 84% of organisations spend less than 5% on evaluation
- 16% spend zero on evaluation (!)
There are also more interesting findings on evaluation use and barriers/supporting factors for evaluation – view the report here (pdf)>>
New resource: Evaluation of Humanitarian Action Guide
ALNAP has recently released their Evaluation of Humanitarian Action Guide.
The guide was six years in the making and contains detailed advice and tips on evaluating humanitarian action. Even if your focus is not on evaluating humanitarian activities, Chapter 17 on Communicating and Reporting Findings and Results is well worth a read.
8 golden rules for communication evaluation
The UK government’s Communication Service has produced a framework for evaluating communications (pdf).
The framework provides an overview of an integrated approach to evaluating communication activities and sets out eight golden rules for communication evaluation:
1. Se
t SMART objectives well before the start of your activity
2. Think carefully about who your target audience is when selecting relevant metrics from each of the five disciplines*
3. Ensure you adopt an integrated channel approach when evaluating your communications activity
4. Collect baselines and benchmarks where possible
5. Include a mix of qualitative and quantitative evidence
6. Regularly review performance
7. Act on any insight to drive continuous improvement and inform future planning
8. Make the link between your activity and its impact on your organisational goals or KPIs
*Media, digital, marketing, stakeholder engagement, internal communications
Are there any more to add? I would add the need to integrate evaluation within the daily work of communication professionals – so it is thought about before starting activities and during…
View the complete guide here (pdf)>>
Useful tool: checklist for quality of evidence
I came across this checklist tool (pdf) from BOND, the UK NGO network on quality of evidence in evaluation. I find the checklist a useful way of…well…checking…an evaluation report to assess it’s quality of evidence. It’s based on five principles: voice and inclusion, appropriateness, triangulation, contribution and transparency. As an evaluator, I will try using it myself to “check” the evaluation reports I author…
View the checklist here (pdf)>>
intelligent measurement 