Priorities for research in public relations
Tom Watson is undertaking a study into the priorities for research in public relations and would like input from communicators via his blog.
If you go there, you’ll find a posting title, “What are the priorities for future public relations research” It asks you to nominate up to 10 topics from a list of 24 (and to add others if you consider there are gaps). The aims of this study, which includes academics, practitioners and the leaders of PR professional and industry bodies around the world, are to:
1. Identify the gaps between academic research and the industry’s needs
2. Identify current and new trends in research from theorist and practitioner perspectives.
3. Map out the best way for knowledge to be exchanged.
4. Define the research areas for which funding can be sought.
This is the first step of a project to consider ideas on how best to promote knowledge exchange between academics and practitioners. Please do participate – I already did, measurement and evaluation of PR activities, particularly social media, figured highly on my list – but you probably could have guessed that :~}
Glenn
Fact Sheets on Communications Evaluation
As part of a breakfast meeting recently held in Geneva on evaluation and communications (where Tom Watson spoke at), I put together a series of fact sheets which some of you may find of interest:
- Introduction to communications evaluation (pdf)
- Evaluating networks (pdf)
- Evaluating communication campaigns (pdf)
Glenn
Macro or Micro Approach to Evaluation?
When I’m asked to take on an evaluation project, I usually categorise it in my own mind as “micro” or “macro”. Let me explain. I see evaluation projects falling into these two categories:
Macro: evaluation of an overall project or programme (e.g. training programme, communications project)
Micro: evaluation of an element that is part of a larger programme (e.g. evaluation of online communications – part of larger communications project, evaluation of an event that is part of larger campaign).
I find that a lot of evaluation projects that I get involved with are at a “micro” level. And I’ve been wondering why is this so?
I believe that evaluation is more often approached at a “micro” level because it is easier to deal with and less daunting for an organisation to cope with. Many people do not have the resources, time and political authority to launch a “macro” evaluation of projects/programmes.
And a lot of the literature on evaluation recommends implementing evaluation aspects in small steps. And there’s certainly some merit in this – to start at the “micro” level and build up to the “macro” level. In this way, people can hopefully see the benefits of evaluation and will support larger evaluation efforts when needed.
Glenn
Online PR Evaluation – do we need new models?
An interesting post here from Tom Watson on his new blog Dummyspit. Tom poses the question – do we need new models of measurement for the emerging social media? I certainly believe we do as I’ve written about previously.
As Tom points out, most current measurement proposed is based on “message distribution” and is an adaptation of what has been proposed for many decades in classic media measurement – an “output” measurement.
Tom goes further and proposes that “out-takes” – the audience reaction to and processing of messages – may be the most appropriate measure of effectiveness. Food for thought..
Glenn
Social Media Measurement
As organisations increasingly look towards the so-called “social media” (which most people usually interpret to mean blogs, wikis, mash-ups, 3D environments etc.) for supporting communications, marketing and other projects, there will certainly be questions asked such as “well, what’s it all worth?”. And that’s fair enough.
Of course, someone is already thinking about it. Constantin Basturea has put together a wiki page (what else?) on social media measurement (found on the NewPRwiki). A brief perusal of the page will show that a lot of the resources are focused on the monitoring aspect – how are publics using social media tools and how companies can track “buzz” or interest. All very good and well, but even more interesting (and substantial) is what is the impact of these social media tools – what do they change? There are some pointers in this direction, I liked very much this photo taken from a workshop that lists what attributes the participants believed should be measured: top of the list is “participation & engagement”. Interesting….
Glenn
Network Mapping – Commercial application?
I read of interest on this post about a network mapping service that has created relational maps about investors, companies and people in the Silicon Valley.
For example, they made a map of the capital links between the three big social networking sites (Facebook, Friendster & LinkedIn). You can view more examples here.
What’s interesting is that this is a paying service – to fully access and use the maps – which points out the value that network mapping can have – for analysing complex situations.
Thanks to Kushtrim Xhakli for bringing this to my attention.
Glenn
Workshop participation & short term impact
An interest of mine is looking at the short & long term impact of conferences and workshops. A lot of work has been done on evaluating the impact of training that I have written about before. Basically, we can look at four levels of impact: 1. Reaction, 2. Learning, 3. Behavior & 4. Results. A lot of conference/workshop evaluation focus on the “reaction” aspect – what did participants like/prefer about an event.
But more interesting is to look at the learning, behavior and- if possible – results aspect. This usually takes time – however, if we are clear about what a workshop/conference is trying to achieve, we can often identify changes in learning/behavior in the short term.
A practical example. When I ran the “Do-It-Yourself Monitoring and Evaluation” workshop (pictured above that’s – David Washburn and myself at the workshop) at the LIFT07 conference, my main objective was to get people thinking about how they could integrate monitoring and evaluation into their own projects. Using a basic evaluation framework (pdf) groups worked to break down projects into the main steps needed for evaluation.
So was the “learning” aspect succesful? – I’d like to think so. Quite a few people commented to me how it got them thinking about monitoring/evaluation and what they could do with their own projects. Also, the following participants blogged about the workshop, an indication of what they took away from the workshop – and also crossing into the “behaviour” area: they processed some thoughts and took the action (behaviour) of writing about it:
- Internet Next Generation blog (David Washburn)>>
- The world we live in (Victoria Visser) >>
- Profession web.ch >>
- Chiperoni.org >>
Even more so, one participant told me about how he used the information from the workshop the same week, which supports my idea about the possiblity to identify short term impact, even in terms of behaviour:
“When we got back from the workshop, I took out the evaluation framework and sat down with my colleagues and planned out what we were going to monitor and evaluate in our major projects, setting down objectives and evaluation indicators. So we can use the framework as a guide in the coming six months.”
Glenn
Beware: dodgy Blog ROI in circulation
Forrester Research have published a new report on the “ROI of blogging” (at USD $ 379 a pop). And I’ve seen that many bloggers have jumped on this with utmost enthusiasm.
Well hold on….
Although Charlene Li of Forrester explains well the ROI model there are some fundamental flaws of the ROI calculation that KD Paine and David Phillips explain further. As KD Paine put it:
“The false assumptions and inaccuracies in this report are scary”
What is the main flaw? Well, the whole blog ROI calculation falls down as it is based on comparing purchased advertising to editorial content, which is a highly discredited way of measuring PR value (read more about comparing advertising to editorial content in this report (pdf) by some leading scholars).
The report does have some interesting points in that it attempts to pull out some of benefits of blogging (such as customer insights) and comparing this to the cost of market research). Certainly the idea of showing how visibility grows from a blog post (through generating comments, thoughts and referrals) to changes in attitudes and behaviors is heading in the right direction, as I’ve written about before.
And as for blogging ROI, I would look more at the cost of working hours in blogging and comparing it to working hours needed to mount a traditional campaign – and comparing the changes to behaviour and attitude using both methods (admittedly easier said than done). That would be more a measure of “efficiency” than anything else.
Glenn
Using employee opinion surveys to guide HR policies
In a previous post Glenn wrote that evaluation is frustrating when no changes result.
Quite so. I have seen many an employee survey gather dust or been quietly forgotten.
Why should employees bother? Often the survey will avoid touchy issues- “The elephant in the room”. There is often a huge delay in making the survey results public. And nothing changes anyway.
Well, here’s a little case history to cast a ray of hope.
A Benchpoint client in the City of London’s financial sector (often characterised by macho management cultures) wanted to run an anonymous poll of employee attitudes on just about everything relating to the job and the relationship with the employer.
The company had very enlightened HR policies by traditional City standards. A “listening culture”, personal development goals, work/life balance, continuous personal feedback etc. But were the policies working? What needed to change?
More than “just a survey”, The Benchpoint poll enabled important decisions to be taken quickly with the benefit of employee input. The company announced operational changes just 5 days after the survey closed, and immediately after a presentation of the results to all employees.
Employee response and feedback was positive; just one employee was unable to take part. For the first time they had an opportunity to voice their true opinions anonymously and confidentially. And they were impressed by the speed of their management’s reaction.
For management, this was a relatively low-cost exercise, and was particularly economic of senior people’s time. It was also an excellent way of “walking the talk” and demonstrating that HR policies regarding inclusivity, development and teamwork were real.
The data collected provided an insight into the behaviour drivers of groups of employees, and a dynamic tool for future goal setting and benchmarking.
Conclusions
- Don’t do a survey unless you are prepared to communicate the results and make changes.
- Don’t make organisation or operational changes unless you have surveyed and understood the real issues.
- Do it quickly, share the results quickly and make your announcements quickly.
- Don’t assume that everything is OK in your organisation. Measure intelligently to find out what the real issues are.
Richard
D.I.Y. Workshop on Monitoring and Evaluation in Geneva
As part of the LIFT07 conference in Geneva, I ran a workshop on “Do-It-Yourself Monitoring and Evaluation” today. Some 40 people attended and the participants did some interesting work in designing evaluation frameworks for their projects. My presentation can be found here:
D.I.Y. Monitoring and Evaluation (pdf – 690 kb)
There was also some interest in the following areas:
- Network mapping; Rick Davies has an excellent web page on this methodology.
- Relationship measurement; a good start are the IPR guidelines.
- Most Significant Change Method and Outcome Mapping are described on this web page quite well (scroll down and you can download the pfd with more details).
- And there was a lot of interest in Blog Return-on-investment. You can read about the work of Forrester Research in this area, but you’d be wise to read the counterpoints on the blogs of K D Paine and LeverWealth.
Glenn
intelligent measurement 