PR Guru Grunig in Switzerland
Professor James Grunig, a leading thinker in the field of communications / PR evaluation will be in Zurich on 16 March 2006 for the MScom Excellence-in-Communications Lecture. James Grunig was one of the first academics to produce credible theory on how communication activities can be evaluated. He later lead the IABC excellence study which focused on the role of PR in strategic management.
If you are interested, you can register on the MScom website .
Disclosure: I am an alumni of the MScom course at the University of Lugano (where I earned my Masters in Communication Management).
Glenn
Context and Evaluation
I noticed recently in an evaluation the influence that the context can have on the findings. Interviewing and surveying people from different countries, we could see how their responses were influenced by their different frames of reference.
If we are aware of context issues (e.g. the setting for a training seminar, the relationship between the respondents and the commissioning organisation, the political climate, etc.) we can better estimate their impact on the findings. Plus it helps see how we can apply our findings to other contexts.
Most of the recognised standards on evaluation speak of the importance of analyzing the influence of context as a question of “accuracy”. Consult the Program Evaluation Standards of the Evaluation Center of Western Michigan University if you are interested to learn more about context and accuracy in evaluation.
Glenn
LIFT 06: Life, Ideas, Future, Together, February 2-3 2006 Geneva
I am glad to announce that Benchpoint is the evaluation partner for LIFT 06, a conference about new technologies and people taking place in Geneva from February 2-3 2006.
We will be involved in evaluating the impact of the event. This will be very interesting as we are using traditional methods and experimenting with how we can use wikis, mashups and blogs to contribute to evaluation.
We will post any findings of note on this blog.
Glenn
Evaluating Communication Campaigns
As the whole development and humanitarian sector focus more on accountability and performance, there has been a push for more evaluation of communication activities of this sector.
Most methodology and tools can be adapted from those used in the private sector. However, many communication campaigns of NGOs and international organisations often have dual outcomes they wish to achieve – individual behaviour change (e.g. persuade individuals to adopt a more healthier lifestyle) and policy change (e.g. push governments to change policy on food labeling).
An excellent study “Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating campaigns ranging from gun safety to emmissions (ozone) reduction.
If you are interested to read more about standards and practices of evaluation in the development and humanitarian sector, a good starting point is The Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP). This interagency forum focuses on evaluation and accountability issues in this sector.
Glenn
Evaluation and Training
In the past decades organisations have invested heavily in training staff often with the aim of increasing productivity. Yet most organisations never evaluate the usefulness of such training. I’m suprised how many organisations still don’t undertake even basic evaluation to measure the reaction of participants.
The most well known model for training evaluation was developed by Donald Kirkpatrick way back in the 70s. But it’s still relevant today – he focused on evaluating four levels of impact:
- Reaction
- Learning
- Behavior
- Results
Reaction measures how well the participants liked the training program and is usually undertaking directly after the training via interviews or surveys. Learning focuses on what has been understood and absorbed once the training is completed through follow-up survey, tests and other methods. Behaviour looks at how the skills and knowledge learnt are used in the work environment and can be measured through interviews (peers, superiors) observations and diaries/logs. Results focuses on the impact on the organisation. This is the hardest step to measure, evidently. The main difficulty is how to determine if any changes in an organisation (e.g. increased motivation) is due to the training. A fifth level, Return on Investment was added at a later stage (notably by J. Phillips).
I’m currently working with organisations in evaluating training with a focus on level four results, so I’ll post interesting findings on this blog.
Meanwhile, if your like to learn more about these four steps of training evaluation, consult this article by none less than four specialists.
Glenn
New to PR Evaluation?
Evaluation of public relations / communications program has been an interest of mine for some time. For those that want to learn more, there are many excellent resources available online. Here are a few I highly recommend:
- Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities from the Institute for PR. This is the hymn book from which we are all singing – it provides key standards for communication measurement that most PR professionals follow.
- PR Measurement on the newPR Wiki. A discussion space with useful resources for those interested in PR measurement.
- Presentations from the 2005 Measurement Summit. Very interesting presentations from the US Public Relations Measurement Summit, a key annual conference in PR evaluation and measurement.
Glenn
A measured approach for 2006
“The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements and expect me to fit them”
George Bernard Shaw
I take the liberty to quote George Bernard Shaw for my first post – his quote on measurement appeals to me. To “behave sensibly” we need to “measure” or else we are working with the “old” that no longer fits. In a nutshell, it sums up my approach to evaluation and measurement and we hope to bring more ideas, discussions and conversations to this area through this blog.
Glenn
intelligent measurement 