Posts filed under ‘Communication evaluation’
Kidneys, Kylie and effects
This month, the Dutch authorities reported that people registering as organ donors had tripled compared to previous months. What caused this sudden jump in registrations – a fantastic awareness programme?
In fact, they trace the increase to the now infamous “Dutch TV Kidney Hoax“, a reality TV show where real patients in need of a kidney “competed” for one.
From the communications evaluation point of view, it is an interesting example of how a communications activity can bring about a rapid change in behaviour (in this case donor registration) and perhaps one that was not intended.
In evaluating our own communication activities, we should try and identify other factors that could have influenced the change being seen – in the kidney TV hoax it was obvious but it will not be for many of the more day-to-day communication activities that we run.
Which reminds me of another example – in August 2005, the number of appointments made for manograms (to detect breast cancer) jumped by 101% in Australia. Was this the result of a successful communications campaign? No, in fact that month, pop singer and fellow Melburnian Kylie Minogue was diagnosed with breast cancer resulting in mass media coverage about the issue which I’ve written about previously.
The identification of other possible explanations for changes being observed (rather than just saying “look our communications campaign worked”) is important in maintaining a credible and balanced approach to evaluation.
Glenn
Measuring online behaviour – statistics to indicators
I’ve written previously about measuring online behaviour and how it can be linked to overall PR evaluation. I found of interest the recent news from Nielsen that they will now rank websites by time spent on sites rather than number of pages viewed. Interesting, as this is a recognition that an indirect indication of “interest” or “engagement” is the amount of time spent on a website, e.g. watching a video, clicking through a slide presentation, reading a text, etc.
When looking at measuring online behaviour, I’ve seen quite some organisations simply drowning in data from web metric software packages and are unable to pull out a real analysis of what they have achieved – or not through the web.
Ultimately indicators should be set to measure success by. These could be:
- “engagement” (average time spent on website),
- “interest” (number of podcast downloaded),
- “conversion” (number of sign-ups for a sales offer),
- “preferences” (growth in visits to a new language version) ,
- etc., etc .
On a related note, when thinking about how to measure online social networking, the Measurement Standard blog provides an interesting list of suggested indicators to measure.
Glenn
New blogs on PR measurement
Two new blogs that have come to my attention recently and focus on public relations and evaluation/measurement:
Evaluating the media from Michael Blowers in the UK
Measurement PRoponent / PRomulgator from Alan Chumley in Canada
Glenn
Download Free PR Measurement Book Now!
No that’s not a spam title there… PR measurement guru KD Paine has put her thoughts down on paper for us and into a 177 page book – which you can download a draft of here in pdf format (1.5 mb). I’ve just been skimming through it and there are certainly some interesting chapters – on measuring relationships with communities, investors and others. K.D Paine welcomes your feedback on her book blog.
Glenn
The problem with ROI
I’ve written previously about Return on Investment (ROI) and particularly its application to blogging (which I consider as flawed).
At a broader level, there has been discussion for some time on ROI for public relations/communications programmes. Tom Watson of the dummyspit blog has written about this issue and the difficulty of applying what is essentially a financial concept to a non-financial activity, as is the case for marketing and PR.
As he states.
“For marketers, the application of ROI limits their role to sales support and ignores the brand and reputational issues. In PR, I’ve long argued that the use of business language is a fundamental sign of insecurity and a lack of confidence. It seems that marketing has the same affliction.”
Glenn
Priorities for research in public relations
Tom Watson is undertaking a study into the priorities for research in public relations and would like input from communicators via his blog.
If you go there, you’ll find a posting title, “What are the priorities for future public relations research” It asks you to nominate up to 10 topics from a list of 24 (and to add others if you consider there are gaps). The aims of this study, which includes academics, practitioners and the leaders of PR professional and industry bodies around the world, are to:
1. Identify the gaps between academic research and the industry’s needs
2. Identify current and new trends in research from theorist and practitioner perspectives.
3. Map out the best way for knowledge to be exchanged.
4. Define the research areas for which funding can be sought.
This is the first step of a project to consider ideas on how best to promote knowledge exchange between academics and practitioners. Please do participate – I already did, measurement and evaluation of PR activities, particularly social media, figured highly on my list – but you probably could have guessed that :~}
Glenn
Fact Sheets on Communications Evaluation
As part of a breakfast meeting recently held in Geneva on evaluation and communications (where Tom Watson spoke at), I put together a series of fact sheets which some of you may find of interest:
- Introduction to communications evaluation (pdf)
- Evaluating networks (pdf)
- Evaluating communication campaigns (pdf)
Glenn
Evaluating Advocacy Campaigns
I’ve written previously about work that others and myself have done on evaluating communication campaigns, particulary concerning campaigns that aim for both changes in individual behaviour and government/private sector policies. In this same direction, a post from the Mission Measurement blog caught my eye on evaluating advocacy campaigns. They make the very relevant point that although evaluating the impact of advocacy campaigns is difficult – trying to isolate the precise influence on changes being observed – what certainly can be measured is the progress towards the desired change.
They go on to provide some further insight into this issue, by looking at various measurements undertaken, such as:
- Number of contacts established
- Intermediate changes to knowledge/attitudes
- Measuring progress of change on a continuum
- Bellweather ratings
In the same vein, what I recommend to organisations is to set clear objectives to start with in terms of what is precisely expected from advocacy/campaigning and establish relatively simple “tracking mechanisms” to follow “progress” on an issue – on a policy level (e..g. number of governments that publicly commit to a given issue) or at an individual level (e.g number of people who pledge to undertake a given action). Often this information is “known” within an organisation but is not centralised or analysed – making any conclusion on a campaign’s impact difficult.
Glenn
Measuring an organisation’s position
At a meeting of communicators working in the health field, I was asked about some of the methodology concerning measuring the “position” of an organisation. As organisations frequently speak of “re-positioning” themselves, it is perhaps wise to know what is your current “position” before you move anywhere…
My approach is relatively simplistic but drawn from the research of the great thinkers in this field, J. Grunig, C. Fombrun and C. van Riel. Basically when we speak of an organisation’s “position”, for me I interpret this to mean what attributes we use to describe an organisation (e.g. modern, creative, traditional, research-focused, event-focused, etc.). This could also be interpreted as the “identity” of the organisation.
Thus to determine a “position” of an organisation, a good starting point would be to question management of an organisation about what attributes they believe are important for the organisation. Then, the main external target publics can be surveyed on what are the main attributes they attribute to the given organisation. Comparing the view of the management to the main target publics can be interesting as certain “gaps” will emerge between how the management view the organisation…and how the organisation is viewed externally. To go one step further, “positioning” would be determined by looking at how similiar or competing organisation are perceived on the same type of attributes.
Actually, what I describe is the basis for most “identity”, “reputation” or “positioning” studies. There is usually a measure of attributes/values/associations internally and a measure of the same externally. And often done with similar/competing organisations to provide a comparison point.
Some resources on this issue:
The book “Principles of Corporate Communications” by C. van Riel has very good descriptions of the main methodologies in identity measurement (ISDN 0131509969)
The Gauge newsletter discussing reputation measurement (pdf) >>
“Corporate reputations can be measured and managed” by C Fombrun (pdf) >>
“Methodology for assessing corporate values” by J. van Rekom, C. van riel & B. Wierenga (pdf) >>
And finally, an interesting opinion on the “demise of positioning“. Certainly some valid points there that I agree with – such as most “re-positioning” campaigns fail – notably because changing a logo, font or slogan doesn’t normally change the behaviour of an organisation – a main influence on “position/brand/identity”. But that’s a whole other issue I’d rather not get into…
Glenn
Measuring Results and Establishing Value
Jenny Schade has an interesting article “Measuring Results and Establishing Value” on her website. The article focuses on how to establish effective metrics and determining measures of success in communications programmes:
At the end of the day, how do you know you’ve been successful? What value are you providing to your organization or clients? In today’s climate of budget cuts and lay-offs, it’s particularly important that you establish clear measures of success before embarking upon any venture.
Read the full article here >>
Glenn
intelligent measurement 