Posts filed under ‘Communication evaluation’
PR Measurement – Generally Accepted Practices Study
An interesting study has been released by the USC Annenberg Strategic Public Relations Center based in California, USA. The “Generally Accepted Practices” study focuses on budgeting, staffing, evaluation and the use of agencies by communication professionals. Some 500 US-based communication professionals responded to the study survey.
An interesting result of the study is the top three evaluation methodologies used by professionals:
1- Influence on corporate reputation
2-Content Analysis of Clips
3-Influence on employee attitudes
The first result is interesting as although there has been methodologies developed on measuring corporate reputation (notably by the Reputation Institute), there is no generally accepted metric on how communication activities “influence” reputation, as the authors of the study point out. What would be interesting to know is how exactly do the professionals assess the influence of their activities on corporate reputation? Active monitoring, comparison studies (to other potential influences), research/analysis or simple intuition?
The reliance on clips is not surprising, but worrying as the authors point out:
“Despite much discussion and debate, evaluation methods have not advanced beyond various forms of content analysis, which is another way of measuring outputs rather than outcomes. The authors suggest that while content analysis is the state of the media measurement art, it ignores all other public relations functions, thereby reinforcing the notion that PR is nothing more than publicity and media relations. This does a disservice to the increasingly sophisticated and complex nature of the profession. Clearly much more work remains to be done in the field of evaluation.”
Thanks to metricsman blog for bringing this study to my attention, where on this post he also makes an interesting analysis on the results.
Glenn
Top Ten Excuses for not Evaluating
This post at the IABC measurement blog caught my attention, as its author asks the question:
“So why don’t we measure more? Is it budget, competencies, time or the risk of accountability?”
People usually don’t evaluate for various reasons, but the most common excuses I’ve heard are the following:
- “It’s too expensive”.
With the amount of free advice, excellent guidelines and cheap research solutions available, this doesn’t pass anymore. - “I don’t know how to”.
Fair enough, but you can learn a lot yourself without having to engage expensive consultants. - “I’m too busy “doing” to be bothered with measuring”.
Frightening. People love doing things, it’s natural. But sometimes you have to stop and take a step back to see what you have achieved. - “What I’m doing couldn’t possibly be measured”.
Often heard from the Creative Type. People who create their own fonts, too clever campaigns and beautiful artwork that impresses other Creative Types. But my question is – what did you change? - “I don’t see the value of it”.
How else can you judge the value of your work if you don’t attempt to analyse and assess it? - “I’m scared what of what I will find out”.
But I think it will be scarier for you if you don’t evaluate and someone else does. - “People are fed up with giving their opinion”.
I don’t think people are – as I’ve written about before. - “My gut feeling tells me I’m doing a good job”.
There is a certain vogue that says out intuition is often our best call. But research often brings out issues that were not even on your radar. - “All my work is vetoed by the CEO, if s/he’s happy so am I”.
The CEO sees the organisation through the same rose-colored glasses as you do. In PR, it’s your public’s perception of your communication that counts. - “You can’t prove anything anyway”.
You can rarely obtain 100% proof that your programme caused the change seen. But what you do is collect evidence that indicates the role your activity played, as I further explained in this post.
Glenn
PR Measurement: are publics fed up?
People (read: potential clients) often tell me that their publics are fed up filling out online survey, being interview or queried about their opinions. Rubbish I say. People are willing to spend time giving their opinion about a website, service or issues that is important to them.
In a recent online survey of external audiences for an international organisation, 25% of the potential audience responded to the survey. And of those who responded, 44% requested a copy of the results. In an evaluation of an international event, 55% of the participants responded to survey. And of those who responded, 60% requested a copy of the results.
In the survey for the international organisation, 80% of respondents said that the organisation’s website was very important or important for their work. Wouldn’t you like to give an opinion on something that important to you? I would.
My experience is that people are willing to participate in research if it is something that is important to them and if they believe something will be done with the results.
In this article on the simply-communicate.com site, the author points out that:
“In a recent survey of reasons for non-response to employee questionnaires, the biggest driver of non-participation was found to be ‘Nothing would happen as a result'”.
So people do want to see that their opinion is valued. The fact that such a large percentage of people want to receive the evaluation results indicate that they are interested in the subject in question. In addition, I imagine that people are interested to see if what they think corresponds to the norm – and to judge what the organisation will do as a consequence of the evaluation.
Glenn
PR Measurement – New Dictionary
The USA’s Institute for Public Relations, which offers all of its research and publications free on the web, has published a new edition of the Dictionary of Public Relations Measurement and Research (pdf – 198kb), edited by Dr. Don W. Stacks of the University of Miami.
We all measure different things and we measure them all differently (and not always intelligently). Not a good idea if you want to benchmark or compare data.
Don Stacks and his colleagues have done a magnificent job in assembling, codifying and defining most of the common terms we come across when talking about measurement, from “Algorithm” to “Z-score”.
It’s a giant step forward in defining some standard methodologies which everyone can use.
The Institute’s website, http://www.instituteforpr.org also contains a large number of downloadable papers on every aspect of communications measurement and evaluation. Well worth a visit.
Richard
The Ninth Trap of PR Measurement
The people over at Cymfony have put together a list of the “Eight traps to Avoid in PR Measurement“, as follows:
1.) Not doing a media audit or assessment before starting a measurement program.
2.) Not defining standard metrics across your organization.
3.) Limiting metrics and analysis to small number of key pubs or simple messages.
4.) Treating all mention s equally.
5.) Not slicing metrics by different audience segments.
6.) Delayed measurement and reporting.
7.) Not taking blogs and online discussions seriously.
8.) Not demonstrating your success, often.
Some of these traps are relevant to all PR and communication measurement, notably 2, 5 6 & 8. However, the focus is clearly on measuring PR outputs and not PR outcomes.
So I would add a ninth trap:
“9. Focusing only on outputs and not outcomes”
PR outputs measure the amount of exposure an organisation receives. More important is to measure PR outcomes – did the PR activities result in any opinion, attitude or behaviour change amongst the targeted audiences?
Given the focus of the list, it’s not suprising to learn that Cymfony offer products to measure PR output. But they should recognise that good PR measurement goes beyond monitoring the amount of press articles, blog posts or online discussions you generate.
Glenn
New to PR Evaluation?
Evaluation of public relations / communications program has been an interest of mine for some time. For those that want to learn more, there are many excellent resources available online. Here are a few I highly recommend:
- Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities from the Institute for PR. This is the hymn book from which we are all singing – it provides key standards for communication measurement that most PR professionals follow.
- PR Measurement on the newPR Wiki. A discussion space with useful resources for those interested in PR measurement.
- Presentations from the 2005 Measurement Summit. Very interesting presentations from the US Public Relations Measurement Summit, a key annual conference in PR evaluation and measurement.
Glenn
intelligent measurement 