Posts filed under ‘Communication evaluation’
Does Creativity Equal Results?
Last week I gave a presentation on evaluation at the First ISO and IEC Marketing and Communication Forum which took place in Geneva. The forum gathered communications and marketing professionals from all over the world working in the field of standards development.
My presentation focused on some of my favourite topics of evaluation, notably:
- Why don’t marketing/communication professionals evaluate
- The need for clarity in setting marketing/communication objectives
- How low cost evaluation can be undertaken
- The risks of being over creative in communications
On the last point, I used the example of the Got Milk campaign which has been lauded as one of the most visible and creative ad campaigns of all time (notably by the advertising industry). However, did the highly creative ads actually help achieve campaign objectives? That is, to get people to drink more milk? Well, milk consumption continues to decline and the ads have been criticised for not addressing a key concern for teenagers – that they consider milk to be fattening.
And that’s the point I tried to make, that creativity all is well and good – but it has to help communicators achieve their campaign goals – and be measurable.
My full presentation can be downloaded here:
Presentation: Effective Marketing & Communications through Evaluation (pdf – 1 MB)
Glenn
Acknowledgement: the example of the Got Milk campaign comes from the book “The Fall of Advertising and the Rise of PR”, A & L Ries.
To Monitor or Evaluate – or Both?
In the latest issue of KD Paine’s Measurement Standard, you will find an article I wrote, here is an extract:
Many communication professionals approach PR measurement with an end game attitude — as something that’s done once a programme or project is concluded. But this is an error; PR measurement also needs to take place before and during a project.
Glenn
Measuring Relationships: A Future Auditing Measure?
Now this may not seem gripping… but I wanted to talk about future global financial reporting and public company auditing procedures – no, please do read on.
There has been a recent initiative “Global Public Policy Symposium” from all the big accounting / auditing firms and regulators to look at how company financial reporting should be done in the future.
They speak about the value of “intangible assets” that should figure in future reporting:
“The value of many companies resides in various “intangible” assets (such as employee creativity and loyalty, and relationships with suppliers and customers). However information to assess the value of these intangibles is not consistently reported”
If you are really keen, read the full report here (pdf)>>
And they go on to conclude that company reporting should in the future include measures on customer and employee satisfaction – and – relationships with key publics.
The idea of measuring relationships and their value is nothing new: there are a number of studies in this area – with the most well known being the Linder Childers and Jim Grunig guidelines on measuring relationships (although a number of points have recently been disputed by Joy Chia (pdf)).
And let’s not forget that David Phillips was way ahead of the auditors when he wrote in 1992 that relationships should be included as intangible assets of an organisation.
Will the auditors adopt a standard measure for relationships? Well, at least they have plenty of work already to draw from.
Glenn
Linking Media Coverage to Business Outcomes
Can we show a link between media coverage and desired business outcomes? A new study (pdf) from the US-based Institute for Public Relations has some interesting case studies that in several instances illustrate corresponding trends between increased media coverage and a desired business outcome occurring.
For example, they speak of a campaign on the importance of mammograms with the desired business outcome being an increase in the number of relevant medical procedures undertaken. Looking at the number of articles published on the issue and comparing it to the number of medical procedures, a correlation seems to exist. This can be seen in the above graph which shows in blue the number of press articles on the issues and in red the number of medical procedures undertaken (over 2 years in the US).
The authors of the study readily admit that they are making a jump in assuming “cause and effect” but what they are looking for is a “preponderance of evidence” that supports a correlation between media coverage and business outcomes.
What I find interesting is the jump from an output measure (clips) to a business outcome. Further, that they were able to find communication campaigns where a clear link was made between communication objectives and business objectives – as often there is a large gap between these two elements.
Read the full study “Exploring the Link Between Volume of Media Coverage and Business Outcomes”(pdf) By Angela Jeffrey, APR, Dr. David Michaelson, and Dr. Don W. Stacks
Glenn
Metrics: You are what you measure!
An interesting post from the Metrics Man on the “Media Measurement Catch-22” as he puts it. His main point is that “you are what you measure”, in other words, you will focus you efforts on achieving the metrics you set, and further:
If all you measure is media relations (primarily clip tonnage), that is how the PR profession will be valued.
Read the full post and if you are interested to learn more about the concept of metrics and how they influence our work efforts, consult the Hauser and Katz paper “Metrics: You are what you measure!” (pdf)
Glenn
Why aren’t we measuring?
A real gem of a paper here (.doc) by Jim Macnamara, a well-know PR evaluation specialist from Australia. He provides an interesting response to the question “why don’t communication professionals measure more?”:
This is the real reason for lack of commitment to measurement. Most PR practitioners do not proactively use research to measure, either for planning or for evaluation, because in their worldview, it is not relevant. When one focuses on and sees one’s job as producing outputs such as publicity, publications and events, measurement of effects that those outputs might or might not cause is an inconsequential downstream issue – it’s someone else’s concern.
A very interesting conclusion – the focus on production is something I’ve seen a lot – I think there is certainly some truth in what he says.
Read the full article here (.doc).
And thanks to K D Paine for sharing this paper with us.
Glenn
It’s Official: Harold Burson says lack of PR Measurement is no. 1 Obstacle
Last night, I attended a communications forum in Geneva where Harold Burson, the founder of the PR agency Burson & Marsteller spoke (in the photo above, he is on the right and Keith Rockwell from WTO on the left).
In responding to a question from a member of the audience (none other than the public affairs representative from the US Mission) – as to how can communicators measure the effectiveness of their programmes, Mr Burson responded:
“The lack of research by communication professionals is the number one obstacle in the PR field today – people don’t do enough research to evaluate the impact of their activities..”
I agree fully. Then he went on to explain the reason “why”. For Mr Burson, the reason is cost – PR research and measurement is too expensive, he mentioned that often research to evaluate can often cost as much as the activities itself. And that’s where I disagree – PR measurement does not have to be expensive. Most capable communication managers should be able to manage measurement tasks themselves through using low cost media monitoring services, easy-to-use online surveys and innovative methods such as case studies and tracking mechanisms. To get started, check out the guidelines from the Institute for PR. And there are certainly other reasons why communication professionals don’t evaluate.
You can read more about the forum on the Geneva Communicators blog. And you can read more about Mr Burson’s thoughts on his blog (is he the oldest PR blogger at 84 years old..?)
Glenn
A Post-Modern Tale on Evaluation
A tale of an organisation concerned with “perception”: Due to some negative press, the organisation was convinced that this was causing a drop in their reputation and affecting their relationship with key government and political stakeholders. Consequently the organisation was pushing to re-orientate their communication activities to lobbying and campaigning activities aimed at government and political circles.
But before going ahead, the organisation did have some input from others (no, not me – but a “good friend”) who suggested evaluating the perception of the organisation amongst stakeholders. A methodology was drawn up and a survey conducted of the major stakeholders. Lo and behold, the organisation was shocked upon seeing the findings. The results showed that government officials and politicians actually had a very good perception of the organisation. But that other important target groups, notably key partners and staff had a negative perception of the organisation. So now the organisation is re-re-orientating activities towards staff and partners.
This simple but true tale illustrates two points that I consider important for image and evaluation:
1) Your view of how your organisation is perceived is probably false. Your stakeholders do not necessarily have access nor are influenced by the same media as you are.
2) The only way to determine how stakeholders perceive your organisations is by asking them. Don’t base your ideas on “feelings” or what the media are reporting. Go to the source.
And like all tales, it has a moral: Our intuition can often be wrong. We base our decisions on biased information formed by our own “world view”. An objective evaluation can be a solution and can alter, sometimes radically, what we thought of as the “truth”.
Glenn
Measuring Online Behaviour – Part 2
Further to my earlier post on measuring online behaviour, I would recommend this article in Brandweek. The article (which I read about on K D Paine’s blog), explains well the current practices of many companies in tracking online behaviour (particularly linked to online campaigns). It goes in the direction that I think – that is, in the online environment, we can measure behaviour of publics to supplement “offline” measurement.
I encourage companies to focus on performance indicators, that moves away from looking at visit statistics and more into what actions are undertaken by a user when visiting a website, for example: referral (referring a page/issue to a friend), commitment (signing-up or endorsing a given activity) or task completion (completing an action online – e.g. playing a game, requesting information, etc.).
Some point of interest I noted from this article:
– Time spent looking at a web feature is an important measure for some campaigns
– IBM looks at registrations and opt-ins as success measures for campaigns
– The Pharmaceutical industry is increasingly turning to online measurement as more and more patients seek medical information online.
Glenn
PR Measurement – Generally Accepted Practices Study
An interesting study has been released by the USC Annenberg Strategic Public Relations Center based in California, USA. The “Generally Accepted Practices” study focuses on budgeting, staffing, evaluation and the use of agencies by communication professionals. Some 500 US-based communication professionals responded to the study survey.
An interesting result of the study is the top three evaluation methodologies used by professionals:
1- Influence on corporate reputation
2-Content Analysis of Clips
3-Influence on employee attitudes
The first result is interesting as although there has been methodologies developed on measuring corporate reputation (notably by the Reputation Institute), there is no generally accepted metric on how communication activities “influence” reputation, as the authors of the study point out. What would be interesting to know is how exactly do the professionals assess the influence of their activities on corporate reputation? Active monitoring, comparison studies (to other potential influences), research/analysis or simple intuition?
The reliance on clips is not surprising, but worrying as the authors point out:
“Despite much discussion and debate, evaluation methods have not advanced beyond various forms of content analysis, which is another way of measuring outputs rather than outcomes. The authors suggest that while content analysis is the state of the media measurement art, it ignores all other public relations functions, thereby reinforcing the notion that PR is nothing more than publicity and media relations. This does a disservice to the increasingly sophisticated and complex nature of the profession. Clearly much more work remains to be done in the field of evaluation.”
Thanks to metricsman blog for bringing this study to my attention, where on this post he also makes an interesting analysis on the results.
Glenn
intelligent measurement 