Measurement Month: Proving & Improving Communications – Smoke Signals Podcast

As part of AMEC’s Measurement Month, I featured on the Smoke Signals Podcast with Paul Cheal, discussing about the practices and challenges of evaluating communications.

November 19, 2021 at 2:27 pm Leave a comment

Are communicators measuring the right things?

Here is an interesting post summarizing a 2021 roundtable discussion from the US-based Institute of Public Relations on “Are PR Pros [communicators] measuring the right things?”. Some highlights from the discussion:

Read the full post here>>

Katie Paine (Paine Publishing): All the things that we have traditionally measured don’t seem as valuable today.–Long ago, the “Grammy of Communications” went to the person who got the most impressions and reported the most hits or clicks or whatever. That’s not going to happen today. I don’t know a lot of organizations that measure on that. Generally, they’re measuring on some kind of change, whether it’s increased awareness, or increased traffic, or saving lives.

“Elizabeth Rector (Cisco): I think it’s an “and.” I think some of the standard metrics aren’t going away, it’s just multiplied, and then we are asking how they come together. That’s blowing up in my world. And finding comms people…finding people who can have that mindset and that blend, because most of my kind of comms insights people have been very much traditional comms, and some of the outcome, like the reputation and brand and all of that. But as we’re getting into thought leadership, and driving to the website, and views, and paid media that’s all now under comms’ umbrella.”

“Joseph Czabovsky (University of North Carolina): I don’t think the fundamentals have changed. As you were saying Katie, it’s still the different elements of funnel, or whether we’re doing awareness, attitude, or behavior, essentially. I think that’s still the core of what we do and what we measure“.

Read the full post here>>

October 8, 2021 at 7:41 am Leave a comment

Campaign evaluation case study

Here is an interesting case study presenting an evaluation of a European-wide smoking cessation campaign (in other words, trying to get people to stop smoking) that I was involved with. Some interesting findings on campaigns and results in this challenging area.

August 6, 2021 at 7:01 pm Leave a comment

Evaluating Advocacy: Challenges, Methodologies and Solutions

Today I was very happy to be with the European Centre for Public Affairs for a training on advocacy evaluation.  For those who are interested, you can see my presentation slides below and I’ve also listed a range of advocacy evaluation and related resources.

Advocacy evaluation an related resources

Theory of change:
Julia Coffman & Tanya Beer (2015), The Advocacy Strategy Framework; A tool for articulating an advocacy theory of change

UNICEF (2014), Theory of change; methodological briefs 

Advocacy monitoring and evaluation methods/approaches:

Saferworld learning paper (2016), Doing things differently – rethinking monitoring and evaluation to understand change (outcome mapping and harvesting)

Robin Kane et al (2017), Contribution Analysis in Policy Work; Assessing Advocacy’s Influence

Harvard Family Research Project (2009), A User’s Guide to Advocacy Evaluation Planning

Julia Coffman (2019), Current Advocacy Evaluation Practice, Center for Evaluation Innovation.

Julia Coffman and Ehren Reed (2019), Unique Methods in Advocacy Evaluation

Intrac (2009), Tracking Progress in Advocacy: Why and How to Monitor and Evaluate Advocacy Projects and Programmes

Oxfam, Process Tracing – Draft Protocol

Guidelines:

CARE (2018), Advocacy and Influencing MEL Guidance

UNICEF, Monitoring and evaluating advocacy; companion to the advocacy toolkit

CIPPEC (2011), Learners, practitioners and teachers; Handbook on monitoring, evaluating and managing knowledge for policy influence

ODI (2014), Monitoring and evaluation of policy influence and advocacy

Communicating monitoring and evaluation results:

Glenn O’Neil (2017), A Guide: Integrating Communication in Evaluation

CDC (2013), Evaluation Reporting: A Guide to Help Ensure Use of Evaluation Findings

“Think pieces” on advocacy evaluation:

Bodille Arensman (2019), Advocacy Outcomes Are Not Self-Evident: The Quest for Outcome Identification

Jim Coe and Rhonda Schlangen (2014), The value iceberg: weighing the benefits of advocacy and campaigning

Jim Coe and Rhonda Schlangen (2019), No Royal Road: Finding and Following the Natural Pathways in Advocacy Evaluation

Jim Coe and Juliette Majot (2013), Monitoring, evaluation and learning in NGO advocacy; Findings from Comparative Policy Advocacy MEL Review Project

Annette L. Gardner and Claire D. Brindis (2017), Advocacy and Policy Change Evaluation: Theory and Practice (a book!)

Examples of advocacy evaluation reports:

UNHCR (2019), Evaluation of Effectiveness and Relevance of Advocacy Approaches in Europe

Norwegian Refugee Council (2014), Evaluation of NRC’s 2012-13 protection and advocacy work in the DRC

Oxfam America (2015), Evaluation of global advocacy programme; Global Leaders Empowered to Alleviate Poverty

Norwegian Refugee Council (2015), Evaluation of global advocacy initiative on housing, land and property

Oxfam International (2013), Evaluation of global communications campaign GROW

March 4, 2021 at 12:39 pm 1 comment

Webinar – what PR metric should I measure? 25th November 2020, 12.30 GMT

As part of the AMEC* Measurement Month, I am speaking as part of a webinar, please join us:

Webinar topic: Ask the Expert: What PR metrics should I measure?

Summary: Join members of the AMEC European chapter for a interactive Q&A discussion on the metrics that really matter. Ask our panel of Europe’s leading experts anything you like about metrics. From how they choose what to measure to the pros and cons of different metrics and what makes a good report.

Speakers:

Dr. Glenn O’Neil, Founder, Owl RE
Dr. Maya Koleva, Head of Research and Insight, Commetric
Dr. Martin Löser, Managing Director Insights, Kantar Media

Date: 25th November 2020, 12.30pm GMT

Register here: https://zoom.us/webinar/register/4416028451632/WN_LvsxNzJoS7-dHssPdZu8gw

*AMEC: International association for the measurement and evaluation of communication

November 20, 2020 at 2:57 pm 1 comment

New resource Integrating Communications in Evaluation on Better Evaluation

I am happy to announce that my guidelines “Integrating communications in evaluation” is now available of the Better Evaluation website. If you haven’t visited Better Evaluation, I suggest you do; it’s full of useful resources and helpful advice.

October 20, 2020 at 6:54 am 1 comment

Implications of COVID-19 on evaluation

The ILO have published useful guidelines on “Implications of COVID-19 on evaluations in the ILO: Practical tips on adapting to the situation“. The guidelines are well worth a read as they can provide guidance for many of us carrying out evaluations remotely these days.
covid-19_

September 1, 2020 at 9:03 am 1 comment

Barcelona Principles 3.0 – communications evaluation

AMEC-Barcelona-Principles-3-High-Res

The Barcelona Principles – which set out the main principles for communication evaluation –  have been updated – version 3.0.

Following is an explanation provided by AMEC.

1. Setting goals is an absolute prerequisite to communications planning, measurement, and evaluation.
The founding principle of SMART (specific, measurable, actionable, relevant, and time-bound) goals as a foundation for communications planning has been promoted to an essential prerequisite. It pushes measurement and evaluation as a core component of the planning process, articulating target outcomes and how progress towards these will be assessed.

2. Measurement and evaluation should identify outputs, outcomes, and potential impact.
Previously, the Principles recommended measuring outcomes, rather than simply counting outputs. The updated principles extend this to consider longer term impact of communications strategy.

3. Outcomes and impact should be identified for stakeholders, society, and the organization.
From the original focus on business metrics, such as sales and revenue, the 2020 update embraces a more holistic view of performance. It allows the model to be more inclusive of a broader range of organisations and communications roles that are not necessarily profit-driven.

4. Communication measurement and evaluation should include both qualitative and quantitative analysis.
“To understand the full impact of your work, it is crucial that you use the full suite of methods to measure those outcomes,” summarised Ben Levine AMEC Board Director in describing the evolution of this principle to not just quantify but also understand how messages are being received, believed and interpreted.

5. AVEs are not the value of communication.
The message remains consistent and clear; “we continue to believe that AVEs do not demonstrate the value of our work.” It is important that communications measurement and evaluation employs a richer, more nuanced, and multi-faceted approach to understand the impact of communications.

6.Holistic communication measurement and evaluation includes all relevant online and offline channels.
Our founding principle that social media can and should be measured is so obvious today. The 2020 iteration reflects the game-changing shift in social communications’ capabilities, opportunities, and influence, such that all relevant online and offline channels should be measured and evaluated equally. The AMEC measurement framework promotes clarity across earned, owned, shared, and paid channels to ensure consistency in approach towards a common goal.

7. Communication measurement and evaluation are rooted in integrity and transparency to drive learning and insights.
Sound, consistent, and sustained measurement calls for integrity and transparency in recognition of today’s attention to data privacy and stewardship as organisations comply with new regulations, such as GDPR. This is also a statement that measurement isn’t simply about data collection and tracking, but about learning from evaluation and applying insight back into communications planning. It recognises the need to be transparent about the context in which programmes are run and being aware of any bias that may exist in the tools, methodologies and interpretations applied.

July 16, 2020 at 7:09 am Leave a comment

AMEC global virtual summit on communication measurement 2020

Summit-2020AMEC has announced their global virtual summit on communication measurement for 2020:

20/20 FORESIGHT: Measurement Frameworks, Myths and New Horizons

The virtual summit will take place on 8-9 July 2020

Keynote speaker will be Distinguished Professor Jim Macnamara of University of Technology Sydney & LSE – Prof. Macnamara is a specialist in communication evaluation and his research is always very interesting.

Jim will present his latest research that inform his new book Beyond Post-Communication and an evaluation project that showed a €24 million return on investment from communication. In two contrasting case studies, Jim will show the negative impact of disinformation and post-truth and the key causes, compared with a ground-breaking case study in evaluation of corporate communication.

Conference fees:
-Standard attendee £350 + VAT
-AMEC Member £250 +VAT
Industry Partner Member (CIPR, ICCO, IPR, PRCA, PRSA) £250 +VAT
-Government/academics/not-for-profit organisations £200 +VAT
-Enterprise delegate offer (up to 15 delegates) £2000 +VAT

Further information>>

 

June 9, 2020 at 2:42 pm Leave a comment

New resource: Measuring Narrative Change

ORSImpact have produced an in-depth guide on measuring narrative change. The resource will be useful in communication and advocacy evaluation where we are often endeavouring to track and measure changes to the narratives of the pertinent issues. View the guide here>>

7102019_41124_521_Measuring_Narrative_Change_07_2019_Cover_thumbnail

May 28, 2020 at 1:06 pm Leave a comment

Older Posts


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,157 other followers

Categories

Feeds