Posts filed under ‘PR evaluation’
Standardisation of PR Evaluation Metrics?
The UK government’s Central Office of Information (yes, I know it’s rather Orwellian sounding…) has produced a set of metrics for measuring PR campaigns: “Standardisation of PR Evaluation Metrics” (pdf).
Frankly, they are disappointing. The title is deceiving, it should be called “Standardisation of media monitoring metrics”. As that all the document covers – the superficial stuff – “reach”, “Favourability of coverage”, etc. These are “outputs” of PR activities.
But what about metrics for measuring “outcomes”? These don’t get a mention. Well there is an admission but you have to dig deep, they do say:
“It is worth bearing in mind that these standardised core metrics for media evaluation are only one component of any campaign evaluation. It is crucial to agree specific key performance indicators (KPIs) at the outset of a campaign.”
So they admit it, these are metrics for media coverage only. And no guidance is given on these KPIs (that are typically “outcome” level). If you are interested in learning more about metrics for “outcomes”, I’d recommend you start with the excellent guide from the Institute of PR: “Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities” (pdf).
AVE to WMC – A wolf in sheep’s clothes?
The Institute for Public Relations has published a new research paper explaining a new media measurement concept called “Weighted Media Cost”. But is this anything new – or simply the dreaded Ad Value Equivalent (AVE) in disguise – a wolf in sheep’s clothes? PR measurement guru KD Paine certainly thinks so.
I think that any measurement based on media space generated by PR efforts is bound to be flawed and increasingly illrelevant. Why?
- Generating media space is rapidly loosing importance as a PR objective – particularly with the growth of other ways that people can obtain information. These measurements typically look at print media – which is a media with a declining readership base
- Measuring how much media space was generated takes the focus away from the more important objectives to measure – what did PR efforts actually change in terms of knowledge, attitudes and behaviors of target audiences. That’s harder to do, but it’s worth the effort…!
Workshop on communications evaluation
I recently conducted a one day training workshop for the staff of Gellis Communications on communications evaluation. We looked at several aspects including:
- How to evaluate communication programmes, products and campaigns;
- How to use the “theory of change” concept;
- Methods specific to communication evaluation including expert reviews, network mapping and tracking mechanisms;
- Options for reporting evaluation findings;
- Case studies and examples on all of the above.
Gellis Communications and myself are happy to share the presentation slides used during the workshop – just see below (these were combined with practical exercises – write to me if you would like copies)
Award for Excellence in PR Measurement & Evaluation
If you are “into” PR/communication research and measurement, and have used it extensively in one of your PR campaigns this past year, you might want to consider entering the Jack Felton Golden Ruler Award for Excellence in PR Measurement & Evaluation. Entries of all types are welcome – including research using social media! The award recognizes superb examples of research used to support public relations practice. Winners are feted at the Institute for Public Relations Summit on Measurement in October in Portsmouth, NH, and it’s quite a big deal.
Be quick – Entries are due August 15th. Here’s How to Enter, and see these examples of previous winners’ entries: Padilla Speer Beardsley’s Winning Entry 2007 or Shell’s Award Winning Entry 2008 for ideas – and there are more on the site – also good if you want to see some best practices in communication / PR evaluation.
Is it possible to create a ROI on communication?
Is it possible to create a ROI on communication? Only if you pursue truly strategic goals…
Given the huge interest in a workable tool to measure ROI on communication, and the amounts of hot air generated on the subject in the two Berlins, I would like to propose a simple blueprint:
It goes like this:
Calculate the value of whatever it is you are trying to promote, sell or protect and the cost of failure.
From this you allocate a reasonable proportion as a budget, enough to enable success at selling, promoting or protecting, etc., recognising the price of failure. (In other words, if it’s not strategic or crucial, think about whether you really want to do it at all).
Measure your inputs – time, resources, materials.
Measure your outputs qualitatively (reach, tone, influence, prestige)
Measure your outputs quantitatively (impressions, views, footfall etc).
You may need to assign a value to your outputs, because PR is not the only game in town. Other communications disciplines will also have played their part.
Measure your outcomes. Not as tricky as it seems. Did you sell any more product? How many people came to the event? Did you save the chemical plant from closure? What was the value?
Don’t get hung up on opinion shifts, unless the shift contributes directly to the achievement of your strategic goals. If it does, then measure it.
I leave it to someone cleverer than me to calculate the golden ratios and percentages and a single number index that will make this work.
It may not be ROI, but it will be a measurement which could become meaningful.
I quote Katie Paine:
“If everyone is so darn hot on ROI, why aren’t they measuring it? I say the reason is, again, lack of knowledge about measurement and measurement tools. Because, there are measurement tools out there that will provide ROI (albeit, not simple or easy to use). So if they really, truly wanted ROI, then they would use measurement techniques that would allow them to calculate it. But instead, they focus on exposure-based methods like clippings and AVEs, because they are afraid of actually learning what the ROI of their programs is. They might find that their programs yielded considerably less R than the I that was put into them.”
I Agree. It’s because so many communications programmes do not have truly strategic objectives in the first place. “Raise awareness”, Increase visibility”, “improve customer feedback” are not strategic objectives, and it is impossible to calculate the value of success or the cost of failure.
Convince legislators on a particular course of action, convince a population that a plant should or should not be built, sell x million units of a new product, persuade us that a particular engineering material is non-carcinogenic and safe, or that a proprietary medicine works – now those are what I call strategic objectives.
They would all merit very grown up budgets, and it would be a dereliction of duty not to measure whether taxpayers or shareholders funds had been wisely spent.
Richard
Berlin stirs up a hornet’s nest
Did you know there are two Berlins? The other is in New Hampshire, USA, and is where Katie Paine, a *PR measurement guru / blogger / twitterite* is based.
Katie has been busy in Berlin, NH writing up her thoughts on what was said in Berlin, Germany at the recent Measurement Summit, and in particularly on the global measurement survey of communication professionals undertaken by Benchpoint for the Summit.
Highly recommended!
Richard
Measurement is an integral part of PR – global survey
As mentioned on K.D Paine’s blog and the Institute for Public Relations website, a new international survey of PR professionals has found that more practitioners than ever are measuring the effectiveness of their communications programmes.
The survey carried out by Benchpoint for AMEC, the international Association for the Measurement and Evaluation of Communication and the Institute for Public Relations, was presented at the 1st European Summit on Measurement in Berlin held from June 10-12 and attended by nearly 200 delegates from 28 countries.
The survey was carried out amongst a sample of 520 PR professionals internationally.
Key findings were:
- The overwhelming majority of PR professionals, 88%, believe measurement is an integral part of the PR process (70% believe this strongly).
- While 77% of respondents claimed to measure their work compared with 69% in a similar survey five years ago, the survey results show that the PR profession are still not agreed on the best tools and methodologies.
- Measuring ROI (return on investment) on communications is viewed as an achievable goal by the overwhelming majority of professional communicators taking part in the survey. There is, however, very strong agreement that it is possible to calculate ROI on communications, and that demonstrable ROI would enhance the budgets (and status) of PR practitioners.
- PR Professionals still tend to judge their success criteria more by their ability to place material in the media rather than on the impact such coverage might have on shifting opinion, awareness, or moving markets, although there is evidence that this is changing.
- The survey found that the tools used by PR professionals includes press clippings – still the favourite – closely followed by AVEs (Advertising value equivalent) and more rigorous tools including Internal Reviews, Benchmarking, and the use of specialist media evaluation tools. Various forms of opinion polling and focus groups also remain as popular tools.
Read more findings from the study>>
Glenn
Latest trends in communications evaluation
At the AMEC Measurement Summit in Berlin, Richard Gaunt, co-author of this blog (pictured next to Glenn O’Neil, co-author at the Summit), presented some key findings of a survey on evaluation practices amongst communication professionals worldwide (0ver 500 respondents). Some of the key findings were:
- 77% of communication professionals are doing some type of evaluation
- Most communication professionals are measuring “outputs”, using media clipping services to monitor visibility
- Two clear groups emerge amongst communication professionals, those who are interested in measuring “outputs” (distribution, number of media mentions) and those who are interested in measuring “outcomes” (changes to knowledge, attitudes and behaviour).
We will be posting more results in the coming days…
Photo – blog authors Glenn and Richard at the AMEC Measurement Summit – photo by K D Paine, see more photos from the Summit >>
Communication evaluation – recession proof?
In a new report released by the International Association for the Measurement and Evaluation of Communication, one of the findings is that a large majority of clients are demanding greater proof of programme effectiveness, with 77% of AMEC members identifying this trend. At the same time, clients are becoming more price sensitive:
Key results of the study include:
- Increasing client interest in measuring social media reported by 92% of AMEC members;
- Clients becoming more price sensitive (92%);
- 69% of members taking part in the study also report that procurement specialists are becoming more involved in the purchase of measurement and evaluation services.
New guide to evaluating communications for non-profits and foundations
The Communications Network has just published a new guide “Are we there yet? A Communications Evaluation Guide (pdf) “. The guide has been written for communicators working in non-profit organisations and foundations – it contains interesting case studies and useful advice. Download the guide here (pdf)>>
intelligent measurement 