Posts filed under ‘General’
Measurement Month: Proving & Improving Communications – Smoke Signals Podcast
As part of AMEC’s Measurement Month, I featured on the Smoke Signals Podcast with Paul Cheal, discussing about the practices and challenges of evaluating communications.
New resource: Measuring Narrative Change
ORSImpact have produced an in-depth guide on measuring narrative change. The resource will be useful in communication and advocacy evaluation where we are often endeavouring to track and measure changes to the narratives of the pertinent issues. View the guide here>>
Measuring people power
MobLab has carried out a global study on how to measure people power – mobilising publics for campaigns and other initiatives. Here is an extract of the key takeaways
:
” key takeaways
For changemakers, if you’re not already measuring depth of supporter relationships and engagement (vs total numbers, or breadth), this report highlights a few ways to get started. Whether in a grassroots group, social movement, or an established NGO, tracking how committed people are to your mission (and to one another) and facilitating deeper relationships can be a big step forward.
For organisational leaders, this report is an urgent call to support cultures that are curious, innovative, and supporter- or people-centred. The results also clarified the importance of investing in staff and resources needed to build relationships with volunteers, grassroots, and community groups. The majority of the surveyed individuals found the very effort of measuring people power shifted and informed both tactics and strategy, guided day-to-day decision making, and was used to evaluate the past; essentially, to show them what was working and what wasn’t.
For funders, the results underscore the need to support organisations, groups and movements by investing in organising and power building, which don’t come with the same attractive metrics that we typically see in fundraising and digital advocacy. Funders should support both the implementation of sustained organising and supporter relationship building (outside of short term programme objectives) and ongoing development and experimentation in power building or “depth” metrics.”
CART principles for monitoring
Innovations for Poverty Action have developed some useful guidance on activity monitoring and evaluation based on their own CART principles: credible, actionable, responsible, and transportable (see summary graphic below).
Particularly useful for those interested in monitoring which is an issue many organisations find challenging. Read more here>>
Evaluation during COVID19 – infographic
Here an informative infographic from UNDP independent evaluation office on evaluation during COVID19 (view the pdf version here).
Evaluation and COVID19
There have been many useful and interesting articles on how evaluation can adapt and cope with the current COVID19 pandemic. Here is a collection of what I’ve found to date:
Practical tools/advice:
Conducting phone-based surveys during COVID 2019
A quick primer on running online events and meetings
Covid-19 crisis: how to adapt your data collection for monitoring and accountability
Think pieces:
Zenda Ofir: Evaluation in times of COVID19 outbreak
World Bank: Conducting evaluations in times of COVID-19
Chris Lysy: The Evaluation Mindset: Evaluation in a Crisis
M Q Patton: Evaluation Implications of the Coronavirus Global Health Pandemic Emergency
Humanitarian advocacy – an introduction
For those interested in the area of humanitarian work and advocacy, this presentation could be of interest – where I explain what is humanitarian advocacy – its definition, levels, process and challenges.
Originally presented at CERAH as part of their Masters in Humanitarian Action.
Example: mixed methods in evaluation
We often talk about using mixed methods in evaluation but we rarely see examples that go beyond a combination of surveys and interviews. So I wanted to share an example of an evaluation that I thought was a good example of using a variety of methods. I was part of a team (of the Independent Evaluation Office) that carried out an evaluation of knowledge management at the Global Environment Facility.
The methods we used included:
-Semi-structured interviews
-Online surveys
-Comparative study of four organisations
-Meta-analysis of country-level evaluations
-Citation analysis – qualitative and quantitative
The image shows the visualisation of the citation analysis (carried out by Matteo Borzoni) by theme – interesting stuff! I feel that the range of data collected gave us a very solid evidence base for the findings. The report is available publicly and can be viewed here (pdf)>>
Infographics to present evaluation findings
I’ve posted previously about using infographics to summarise evaluation findings; here is a recent example of using an infographic to present research results (click on the image to see larger and complete version); admittedly we packed a lot into this infographic – but still a good summary!
The global rise and future challenges for evaluation
Very interesting article from the latest edition of the newsletter of the European Evaluation Society “Evaluation between evidence‑based policy and “fake news”: paths to the future” (p. 3).
The article presents the reasons why there has been a global rise in evaluations and four future challenges, summarised here:
Why the global rise of evaluation:
1) In many countries, evaluation is a fixed element in policy-shaping and a management control element
2) The number of national evaluation societies has grown
3) Market for evaluation is continuing to grow
4) The dissemination of evaluation findings has surged
5) Training activities have increased
Future challenges for evaluation:
1) Increasing importance of global issues
2) The challenge of populist movements and “fake news” to evidence-based policy
3) Need for reflexive systems-thinking approaches
4) Increasing demand for participatory evaluation
intelligent measurement 