Posts filed under ‘PR evaluation’
Social Media Measurement
As organisations increasingly look towards the so-called “social media” (which most people usually interpret to mean blogs, wikis, mash-ups, 3D environments etc.) for supporting communications, marketing and other projects, there will certainly be questions asked such as “well, what’s it all worth?”. And that’s fair enough.
Of course, someone is already thinking about it. Constantin Basturea has put together a wiki page (what else?) on social media measurement (found on the NewPRwiki). A brief perusal of the page will show that a lot of the resources are focused on the monitoring aspect – how are publics using social media tools and how companies can track “buzz” or interest. All very good and well, but even more interesting (and substantial) is what is the impact of these social media tools – what do they change? There are some pointers in this direction, I liked very much this photo taken from a workshop that lists what attributes the participants believed should be measured: top of the list is “participation & engagement”. Interesting….
Glenn
Beware: dodgy Blog ROI in circulation
Forrester Research have published a new report on the “ROI of blogging” (at USD $ 379 a pop). And I’ve seen that many bloggers have jumped on this with utmost enthusiasm.
Well hold on….
Although Charlene Li of Forrester explains well the ROI model there are some fundamental flaws of the ROI calculation that KD Paine and David Phillips explain further. As KD Paine put it:
“The false assumptions and inaccuracies in this report are scary”
What is the main flaw? Well, the whole blog ROI calculation falls down as it is based on comparing purchased advertising to editorial content, which is a highly discredited way of measuring PR value (read more about comparing advertising to editorial content in this report (pdf) by some leading scholars).
The report does have some interesting points in that it attempts to pull out some of benefits of blogging (such as customer insights) and comparing this to the cost of market research). Certainly the idea of showing how visibility grows from a blog post (through generating comments, thoughts and referrals) to changes in attitudes and behaviors is heading in the right direction, as I’ve written about before.
And as for blogging ROI, I would look more at the cost of working hours in blogging and comparing it to working hours needed to mount a traditional campaign – and comparing the changes to behaviour and attitude using both methods (admittedly easier said than done). That would be more a measure of “efficiency” than anything else.
Glenn
Evaluating Advocacy Campaigns
I’ve written previously about work that others and myself have done on evaluating communication campaigns, particulary concerning campaigns that aim for both changes in individual behaviour and government/private sector policies. In this same direction, a post from the Mission Measurement blog caught my eye on evaluating advocacy campaigns. They make the very relevant point that although evaluating the impact of advocacy campaigns is difficult – trying to isolate the precise influence on changes being observed – what certainly can be measured is the progress towards the desired change.
They go on to provide some further insight into this issue, by looking at various measurements undertaken, such as:
- Number of contacts established
- Intermediate changes to knowledge/attitudes
- Measuring progress of change on a continuum
- Bellweather ratings
In the same vein, what I recommend to organisations is to set clear objectives to start with in terms of what is precisely expected from advocacy/campaigning and establish relatively simple “tracking mechanisms” to follow “progress” on an issue – on a policy level (e..g. number of governments that publicly commit to a given issue) or at an individual level (e.g number of people who pledge to undertake a given action). Often this information is “known” within an organisation but is not centralised or analysed – making any conclusion on a campaign’s impact difficult.
Glenn
Measuring an organisation’s position
At a meeting of communicators working in the health field, I was asked about some of the methodology concerning measuring the “position” of an organisation. As organisations frequently speak of “re-positioning” themselves, it is perhaps wise to know what is your current “position” before you move anywhere…
My approach is relatively simplistic but drawn from the research of the great thinkers in this field, J. Grunig, C. Fombrun and C. van Riel. Basically when we speak of an organisation’s “position”, for me I interpret this to mean what attributes we use to describe an organisation (e.g. modern, creative, traditional, research-focused, event-focused, etc.). This could also be interpreted as the “identity” of the organisation.
Thus to determine a “position” of an organisation, a good starting point would be to question management of an organisation about what attributes they believe are important for the organisation. Then, the main external target publics can be surveyed on what are the main attributes they attribute to the given organisation. Comparing the view of the management to the main target publics can be interesting as certain “gaps” will emerge between how the management view the organisation…and how the organisation is viewed externally. To go one step further, “positioning” would be determined by looking at how similiar or competing organisation are perceived on the same type of attributes.
Actually, what I describe is the basis for most “identity”, “reputation” or “positioning” studies. There is usually a measure of attributes/values/associations internally and a measure of the same externally. And often done with similar/competing organisations to provide a comparison point.
Some resources on this issue:
The book “Principles of Corporate Communications” by C. van Riel has very good descriptions of the main methodologies in identity measurement (ISDN 0131509969)
The Gauge newsletter discussing reputation measurement (pdf) >>
“Corporate reputations can be measured and managed” by C Fombrun (pdf) >>
“Methodology for assessing corporate values” by J. van Rekom, C. van riel & B. Wierenga (pdf) >>
And finally, an interesting opinion on the “demise of positioning“. Certainly some valid points there that I agree with – such as most “re-positioning” campaigns fail – notably because changing a logo, font or slogan doesn’t normally change the behaviour of an organisation – a main influence on “position/brand/identity”. But that’s a whole other issue I’d rather not get into…
Glenn
Six factor to ensure that your evaluation results are used
As I wrote in a previous post, evaluation can be quite frustrating when all your effort and work doesn’t actually lead to any significant change in future practices. Why are evaluations not used? A new report “The Utilisation of Evaluations” (pdf) from ALNAP throws some light on this subject. Although focusing on the humanitarian sector, the report has some points that apply to all types of evaluations. I found interesting the six quality factors the author identifies that contribute to the findings of an evaluation being utilised, notably:
- Designing carefully the purpose and approach of the evaluation
- Managing quality participation of all stakeholders throughout the evaluation
- Allowing enough time to have all relevant staff and stakeholders involved
- Ensuring that the evidence is credible and the report is easy to read with clear, precise recommendations with who is responsible for what and when
- Putting in place follow-up plans at the outset
- Ensuring that the evaluator(s) are credible, balanced and constructive – wholesale negativity is never welcomed
Going through these six factors I can see where I’ve faced obstacles in past evaluations, notably points 2 and 5. I find managing stakeholder involvement is often difficult and so is setting out follow-up plans – it often comes as an after-thought. Certainly some factor to consider for all evaluators…
Read the full report (pdf) here >>
Glenn
Presenting monitoring & evaluation results
The more I work in the M&E field, the more I see the importance of presenting results in a consumable way. If you are leading an evaluation project, there is nothing more frustrating than finishing your project and finding the comprehensive report you wrote gathering dust on a manager’s desk.
But that’s what I have learnt, the comprehensive report will perhaps only be read by one or two people of the commissioning team – but the powerpoint summarising the report will be widely distributed and viewed by many. We may think this is a “dumbing-down” of the work undertaken but it is a reality of how our work is consumed. Here are some points on presenting results that I find useful:
- Think carefully about the data and findings you want to present. We can often be overwhelmed by data (from survey results for example). If in doubt, put data you consider important but not essential in report annexes.
- Make the evaluation report attractive and easy to ready – facilitate this by summarising the main points and creating a brief presentation.
- Organise an event such as a staff or team meeting to discuss the results – this could have more impact than the written document.
- Through blogs and wikis, use the evaluation results to generate more discussion and interest in the given subject. A good example is the blog created to present the results of the 2006 Euroblog survey.
Jim Macnamara in a recent article (pdf) touches on this subject on how presenting results with a “two-tier” approach is useful – that is, presenting to top management only key data and information while fully digesting all data at the corporate communications level.
Glenn
Cartoon from toothpaste for dinner>>
Measuring Results and Establishing Value
Jenny Schade has an interesting article “Measuring Results and Establishing Value” on her website. The article focuses on how to establish effective metrics and determining measures of success in communications programmes:
At the end of the day, how do you know you’ve been successful? What value are you providing to your organization or clients? In today’s climate of budget cuts and lay-offs, it’s particularly important that you establish clear measures of success before embarking upon any venture.
Read the full article here >>
Glenn
Does Creativity Equal Results?
Last week I gave a presentation on evaluation at the First ISO and IEC Marketing and Communication Forum which took place in Geneva. The forum gathered communications and marketing professionals from all over the world working in the field of standards development.
My presentation focused on some of my favourite topics of evaluation, notably:
- Why don’t marketing/communication professionals evaluate
- The need for clarity in setting marketing/communication objectives
- How low cost evaluation can be undertaken
- The risks of being over creative in communications
On the last point, I used the example of the Got Milk campaign which has been lauded as one of the most visible and creative ad campaigns of all time (notably by the advertising industry). However, did the highly creative ads actually help achieve campaign objectives? That is, to get people to drink more milk? Well, milk consumption continues to decline and the ads have been criticised for not addressing a key concern for teenagers – that they consider milk to be fattening.
And that’s the point I tried to make, that creativity all is well and good – but it has to help communicators achieve their campaign goals – and be measurable.
My full presentation can be downloaded here:
Presentation: Effective Marketing & Communications through Evaluation (pdf – 1 MB)
Glenn
Acknowledgement: the example of the Got Milk campaign comes from the book “The Fall of Advertising and the Rise of PR”, A & L Ries.
To Monitor or Evaluate – or Both?
In the latest issue of KD Paine’s Measurement Standard, you will find an article I wrote, here is an extract:
Many communication professionals approach PR measurement with an end game attitude — as something that’s done once a programme or project is concluded. But this is an error; PR measurement also needs to take place before and during a project.
Glenn
Measuring Relationships: A Future Auditing Measure?
Now this may not seem gripping… but I wanted to talk about future global financial reporting and public company auditing procedures – no, please do read on.
There has been a recent initiative “Global Public Policy Symposium” from all the big accounting / auditing firms and regulators to look at how company financial reporting should be done in the future.
They speak about the value of “intangible assets” that should figure in future reporting:
“The value of many companies resides in various “intangible” assets (such as employee creativity and loyalty, and relationships with suppliers and customers). However information to assess the value of these intangibles is not consistently reported”
If you are really keen, read the full report here (pdf)>>
And they go on to conclude that company reporting should in the future include measures on customer and employee satisfaction – and – relationships with key publics.
The idea of measuring relationships and their value is nothing new: there are a number of studies in this area – with the most well known being the Linder Childers and Jim Grunig guidelines on measuring relationships (although a number of points have recently been disputed by Joy Chia (pdf)).
And let’s not forget that David Phillips was way ahead of the auditors when he wrote in 1992 that relationships should be included as intangible assets of an organisation.
Will the auditors adopt a standard measure for relationships? Well, at least they have plenty of work already to draw from.
Glenn
intelligent measurement 