Posts filed under ‘PR evaluation’
Changing behaviour – immediate responses
Adding to what I wrote about last week concerning measuring behaviour changes that result from communication campaigns – and why I recommend to consider looking at immediate responses (or “outtakes”) as an alternative to long-term changes – I can see parallels in areas other than in campaigns.
As you may know, a favourite of mine is measuring the impact of conferences and meetings. Industry conferences are traditionally sold as being great places to learn something and network, network – and network. But I’m always surprised when attending such conferences at how organisers, if they measure something, focus on measuring the reactions to the conferences, usually in terms of satisfaction. No attempt is made to measure immediate changes to behaviour (such as extending a network) or longer term behaviour or impact in general.
But it is certainly possible, this diagram (pdf) illustrates what I did to measure immediate and mid-term changes to behaviour following a conference (LIFT). Despite the limitations of the research as I explain here, I was able to track some responses following the conference that could be largely contributed to participating in the conference – such as meeting new people or using new social media in their work. One year after the conference, participants also provided us with types of actions that they believed were influenced largely by their participation. Actions included:
– launching a new project
– launching a new service/product
– establishing a new partnership
– Initiating a career change
– Invitations for speaking engagements
Some of these actions were anticipated by the conference organisers – but many were not. It shows that it can be done and is certainly worth thinking about in conference evaluation.
Glenn
Changing behaviour – takes a long time?
In an interesting post by Tom Watson on the Dummyspit blog, he writes about the difficulty of changing behaviour through PR campaigns – in this case to encourage better eating habits in UK schools (staying away from hamburgers for example). He also notes the rather depressing statistic that PR campaigns normally have only a 0.04% success rate of changing behaviour! In setting campaign objectives, I encourage organisations to be realistic about the targets they set – for example, a modest 2 – 10% behaviour change for public health campaigns is cited as being a realistic target (Rice & Paisley 1982).
But before you abandon your PR activities in despair, we have to be clear about what we define as “behaviour change”. Changing eating habits is a long-term behaviour change. PR campaigns often focus on trying to change long term behaviour and don’t consider setting goals in terms of immediate responses – what can people be expected to do when reached by the campaign? In PR terminology this is called “outtakes” (different from long-term behaviour which would be “outcomes”).
Following are some practical examples of “outtakes” or immediate responses (which are short term behaviours) that can be measured. I believe that the percentage of change desired amongst a target audience can can be set at higher than 10% for many of these:
– % of people that sign a petition
– % of people that refer a web page to a friend
– % people that go online to participate in an online discussion
– % of organisations/individuals that publicly support a campaign
– % of people who report undertaking a new initiative as a result of a campaign
Glenn
Kidneys, Kylie and effects
This month, the Dutch authorities reported that people registering as organ donors had tripled compared to previous months. What caused this sudden jump in registrations – a fantastic awareness programme?
In fact, they trace the increase to the now infamous “Dutch TV Kidney Hoax“, a reality TV show where real patients in need of a kidney “competed” for one.
From the communications evaluation point of view, it is an interesting example of how a communications activity can bring about a rapid change in behaviour (in this case donor registration) and perhaps one that was not intended.
In evaluating our own communication activities, we should try and identify other factors that could have influenced the change being seen – in the kidney TV hoax it was obvious but it will not be for many of the more day-to-day communication activities that we run.
Which reminds me of another example – in August 2005, the number of appointments made for manograms (to detect breast cancer) jumped by 101% in Australia. Was this the result of a successful communications campaign? No, in fact that month, pop singer and fellow Melburnian Kylie Minogue was diagnosed with breast cancer resulting in mass media coverage about the issue which I’ve written about previously.
The identification of other possible explanations for changes being observed (rather than just saying “look our communications campaign worked”) is important in maintaining a credible and balanced approach to evaluation.
Glenn
Measuring online behaviour – statistics to indicators
I’ve written previously about measuring online behaviour and how it can be linked to overall PR evaluation. I found of interest the recent news from Nielsen that they will now rank websites by time spent on sites rather than number of pages viewed. Interesting, as this is a recognition that an indirect indication of “interest” or “engagement” is the amount of time spent on a website, e.g. watching a video, clicking through a slide presentation, reading a text, etc.
When looking at measuring online behaviour, I’ve seen quite some organisations simply drowning in data from web metric software packages and are unable to pull out a real analysis of what they have achieved – or not through the web.
Ultimately indicators should be set to measure success by. These could be:
- “engagement” (average time spent on website),
- “interest” (number of podcast downloaded),
- “conversion” (number of sign-ups for a sales offer),
- “preferences” (growth in visits to a new language version) ,
- etc., etc .
On a related note, when thinking about how to measure online social networking, the Measurement Standard blog provides an interesting list of suggested indicators to measure.
Glenn
New blogs on PR measurement
Two new blogs that have come to my attention recently and focus on public relations and evaluation/measurement:
Evaluating the media from Michael Blowers in the UK
Measurement PRoponent / PRomulgator from Alan Chumley in Canada
Glenn
Download Free PR Measurement Book Now!
No that’s not a spam title there… PR measurement guru KD Paine has put her thoughts down on paper for us and into a 177 page book – which you can download a draft of here in pdf format (1.5 mb). I’ve just been skimming through it and there are certainly some interesting chapters – on measuring relationships with communities, investors and others. K.D Paine welcomes your feedback on her book blog.
Glenn
The problem with ROI
I’ve written previously about Return on Investment (ROI) and particularly its application to blogging (which I consider as flawed).
At a broader level, there has been discussion for some time on ROI for public relations/communications programmes. Tom Watson of the dummyspit blog has written about this issue and the difficulty of applying what is essentially a financial concept to a non-financial activity, as is the case for marketing and PR.
As he states.
“For marketers, the application of ROI limits their role to sales support and ignores the brand and reputational issues. In PR, I’ve long argued that the use of business language is a fundamental sign of insecurity and a lack of confidence. It seems that marketing has the same affliction.”
Glenn
Priorities for research in public relations
Tom Watson is undertaking a study into the priorities for research in public relations and would like input from communicators via his blog.
If you go there, you’ll find a posting title, “What are the priorities for future public relations research” It asks you to nominate up to 10 topics from a list of 24 (and to add others if you consider there are gaps). The aims of this study, which includes academics, practitioners and the leaders of PR professional and industry bodies around the world, are to:
1. Identify the gaps between academic research and the industry’s needs
2. Identify current and new trends in research from theorist and practitioner perspectives.
3. Map out the best way for knowledge to be exchanged.
4. Define the research areas for which funding can be sought.
This is the first step of a project to consider ideas on how best to promote knowledge exchange between academics and practitioners. Please do participate – I already did, measurement and evaluation of PR activities, particularly social media, figured highly on my list – but you probably could have guessed that :~}
Glenn
Fact Sheets on Communications Evaluation
As part of a breakfast meeting recently held in Geneva on evaluation and communications (where Tom Watson spoke at), I put together a series of fact sheets which some of you may find of interest:
- Introduction to communications evaluation (pdf)
- Evaluating networks (pdf)
- Evaluating communication campaigns (pdf)
Glenn
Online PR Evaluation – do we need new models?
An interesting post here from Tom Watson on his new blog Dummyspit. Tom poses the question – do we need new models of measurement for the emerging social media? I certainly believe we do as I’ve written about previously.
As Tom points out, most current measurement proposed is based on “message distribution” and is an adaptation of what has been proposed for many decades in classic media measurement – an “output” measurement.
Tom goes further and proposes that “out-takes” – the audience reaction to and processing of messages – may be the most appropriate measure of effectiveness. Food for thought..
Glenn
intelligent measurement 