Posts tagged ‘communication evaluation’
Measurement Month: Proving & Improving Communications – Smoke Signals Podcast
As part of AMEC’s Measurement Month, I featured on the Smoke Signals Podcast with Paul Cheal, discussing about the practices and challenges of evaluating communications.
New resource: Communications monitoring, evaluation and learning toolkit
The Uk-based Overseas Development Institute (ODI) has published a new resource “Communications monitoring, evaluation and learning toolkit“; the toolkit focuses mainly on how to monitor your communication outputs and has some handy indicators by channel.
8 golden rules for communication evaluation
The UK government’s Communication Service has produced a framework for evaluating communications (pdf).
The framework provides an overview of an integrated approach to evaluating communication activities and sets out eight golden rules for communication evaluation:
1. Se
t SMART objectives well before the start of your activity
2. Think carefully about who your target audience is when selecting relevant metrics from each of the five disciplines*
3. Ensure you adopt an integrated channel approach when evaluating your communications activity
4. Collect baselines and benchmarks where possible
5. Include a mix of qualitative and quantitative evidence
6. Regularly review performance
7. Act on any insight to drive continuous improvement and inform future planning
8. Make the link between your activity and its impact on your organisational goals or KPIs
*Media, digital, marketing, stakeholder engagement, internal communications
Are there any more to add? I would add the need to integrate evaluation within the daily work of communication professionals – so it is thought about before starting activities and during…
View the complete guide here (pdf)>>
communication evaluation event in Zurich – 28 April 2016
For any readers in the Zurich, Switzerland area, I will be giving a presentation for the EMScom Alumni Association (of which I am an alumni of..) on communication evaluation; here is a short description:
Evaluation of communication activities is consistently named as one of the top concerns of communication professionals. Yet paradoxically not even half reportedly undertake any evaluation. Drawing from his recent PhD studies and over a decade of experience in evaluating communication campaigns and programmes, Glenn O’Neil will set out the challenges and complexities of evaluation and offer insights into solutions and approaches to ensure that evaluation brings value to communication professionals and their organisation
Thursday, April 28, 2016, 18h30-21h00
Widder Hotel, Zürich
Cost: 50 CHF (free for EMScom alumni)
Hope to see some of you there! Further information >>
Register also by email: emscomalumni@usi.ch
Glenn
communication evaluation in UN agencies
The UN Joint Inspection Unit (JIU), an external oversight body of all of the United Nations, has just produced a very interesting study on Public information and communications policies and practices in the United Nations system (pdf)
Aside from providing an interesting and critical view of communications in the UN, the report also looks at the monitoring and evaluation (M&E) of communication activities – concluding that M&E needs to better feed into management directions and decision-making. Here are some key findings from the report on M&E and communications:
-Only half of UN agencies have included M&E in their communication frameworks
-Indicators used are predominantly output-based
-There is absence of an evaluation culture among communication staff
– Existing monitoring systems (e.g. for media coverage) were largely descriptive, rarely analysed and did not feed into decision-making.
View the full report here (pdf) – M&E aspects discussed from page 22 onwards.
EU communication evaluation toolkit
The European Commission has released a communication evaluation toolkit (pdf). Produced by the Directorate General for Communication, the kit provides a very good overview of evaluating communication activities and also provides individual fact sheets on evaluating events, social media, publications, newsletters, conferences, websites and even smartphone apps…
View the manual here (pdf)
Interested in more communication evaluation toolkits? see my earlier post for five more toolkits or handbooks.
Evaluating communication – 4 of 4 key lessons
In my last post I wrote about the third of four key lessons on evaluation for communicators.
My first lesson was “1. Planning is key”
My second lesson was “2. Decide what and how you will measure”
My third lesson was: “3. Find the time & budget to monitor and evaluate”
And my forth lesson is: “4. Analyse and share your results”
Too often, evaluation and monitoring data is not analysed; I’ve seen many media monitoring reports, web statistics and event feedback forms not compiled and analysed – communicators should be looking at such data to help understand what has been achieved (or not) and how activities could be improved. Once data is analysed, you should then see how such findings can be shared with your colleagues and stakeholders. Evaluation findings will be of interest to yourself as they concern your activities but they may also be of interest to others in your organisation and possibly externally (confidentiality taken into consideration).
Evaluating communication – 2 of 4 key lessons
In my last post I wrote about the first of four key lessons on evaluation for communicators.
My first lesson was “1. Planning is key”.
My second lesson was “2. Decide what and how you will measure”
If you have planned well your communication actions, then you should have clear and measurable objectives. Your “what” to measure should normally be based at the level of objectives and their outcomes – not at the level of activities. For example, if you have set your objective as ” to put issue XY on the agenda of AB organisation” you should be measuring this notion, that is, is the issue on the agenda of AB- and not the activities and their outputs to reach this, which could be a series of meetings, events, etc.
So if you have a broad communication action you might have to narrow down the “what” you want to measure – of course, I’d recommend going for the more significant – assessing if an issue is on the agenda is more significant than assessing how many people attended a given event contributing to this goal (the former being a “outcome” and the latter being an “output”).
The “how” to measure has perhaps attracted the most attention in communication evaluation. There are plenty of guides and handbooks available that focus on the methods and tools to use to measure, here is a list of several that I find good. Key is to finding the tool/method that suits what you want to measure. For the above example, how could you measure if “issue XY is on the agenda of AB”? This could be done by studying public statements by AB on the given issue and assessing to what extent it is on their agenda.
Next post, lesson 3!
Evaluating communication – 1 of 4 key lessons
last week I had the opportunity to present to a group of communicators from human rights NGOs at a True Heroes Films workshop in Geneva.
One of the main themes I spoke about was on lessons for evaluating communication campaigns and programmes based on my own experiences of having being involved in some 100 evaluations in this field.
I identified four lessons that I believe apply for all communicators, non-profit or for-profit, particularly taking into account the challenges faced of limited resources.
My first lesson was “Planning is key“.
For communicators, planning is often seen as a necessary burden before they get on to the exciting stuff – actually doing things! Communicators often jump straight into doing activities, i.e. setting up a website, organising an event, issuing a press release, etc. without actually fully thinking through the purpose of the activity – why are we doing this?
So to have a clear strategy and plan is key – before starting:
-Does the situation merit to communicate (analysis)?
-What do you want to achieve (objectives)?
-With whom (publics)?
-What?(messages)
-How (activities/tactics)
Thinking about this before you communicate will make evaluation so much easier – and possible!
That was lesson no. 1. And lessons 2-4? Wait for my next posts!
Evaluating communication products – an example
I’ve written previously about the challenge of evaluating communication products – websites, brochures, videos and reports – rather than a communication campaign or programme as a whole.
I’ve just been involved in carrying out an evaluation of products for ACAPS, an NGO focused on assessment in humanitarian crises – and as part of their mandate they produce analytical products (mainly reports) on crisis situations globally. To carry out the evaluation, we undertook 40 interviews with users of their products and then analysed and categorised the type of use, satisfaction ratings and unmet needs. You can view the evaluation report here (pdf)>>.
intelligent measurement 