Online courses in evaluation
Claremont Graduate University (USA) is offering three online courses in evaluation in the next months at very reasonable prices:
* March 28-29: Hard-Core Qualitative Research Methods, Part A (Michael Scriven)
* April 4-5: Hard-Core Qualitative Research Methods, Part B (Michael Scriven)
* May 19-20: Introduction to Practical Program Evaluation: A Theory-Driven Approach (Stewart I. Donaldson)
Each course includes two 3-hour training sessions in an online virtual classroom environment, available to anyone with aninternet connection anywhere in the world.
These courses are offered at USD $90 each.
Full descriptions are available on their website>>
(the authors of this blog have no commercial relationship with the courses – they just seem interesting and at a good price!)
First European Summit on Communication Measurement 10 – 12 June 2009
The first European Summit on Communication Measurement is planned for 10 – 12 June 2009 in Berlin, Germany and the program looks very interesting so far. The summit starts with a full day of workshops and the speakers over the next two days include:
* Bjorn Edlund, VP Communications, Shell International
* Pablo Fernández Calvo, Director of Communication, BBVA
* David Michaelson, President USA, Echo Research
* Neil Martinson, Director of News & PR, UK’s Central Office of Information
* Frank Oviatt, President & CEO of the Institute for Public Relations
* Katie Paine,CEO, KD Paine & Associates
* David Rockland,Partner,Managing Director,Ketchum
* Dr Christopher Storck, Principal, HeringSchuppener
* Dr Tom Watson, Bournemouth University
* Dr Donald K Wright, Boston University
* Professor Dr Ansgar Zerfass, EUPRERA/University of Leipzig
Some of the topics to be covered include:
* Social Media – The Big New Opportunity for Communications
* Traditional Media is Adapting with Confidence to Embrace the Challenge of Social Media
* How Clients see Media Change and Experiments with Communications Models
* International Measurement – an Insight into the Latin American Regional Analysis for BBVA
* In Defence of AVE’s
* What Media Evaluation will look like in the future: the next generation
* Public Relations in Europe – Status Quo and Future Challenges
* The Comparative Communications Effectiveness of Advertising and PR
* Social Media Measurement Case Studies
* The Role of Research in Communications
* Predicting the Future for Communications-What International Research Tells Us
The conference is organised by the International Association for Measurement and Evaluation of Communications , the Institute for Public Relations (US) and the German Public Relations Public Relations Association (DPRG).
Six outcome categories for advocacy campaigns
As I mentioned in an earlier post, an interesting guide on measuring advocacy and policy (pdf) has been published by the Annie E. Casey Foundation. What I found interesting in this guide was that the authors have determined six outcome categories for advocacy campaigns. Studying campaigns, they identified what advocacy campaigns aim to achieve (“outcomes”) and broke it down into six categories:
1) Shift in social norms: e.g. knowledge, values, behaviour of society
2) Strengthened organisational capacity: e.g. skill set and structure of coalitions that carry out advocacy work.
3) Strengthened alliances: e.g. structural changes in community and insitutional relationships and alliances.
4) Strengthened base of support: e.g. grassroots, leadership and institutional support for policy changes.
5) Improved policies: e.g. policy development, demonstration of support, adoption, funding and implementation.
6) Changes in impact: e.g. ultimate change in social and physical lives and conditions.
Read more about these outcomes (go to page 17) in the guide (pdf).
Measuring social media – Twitter
For those interested in measuring social media, here is an interesting tool to monitor Twitter and its users – the Twitter influence calculator. You can discover the most influential users and compare yourself to others. Plus, they present the information in graphs and tag clouds as seen above.
Glenn
Guide to measuring advocacy and policy
Here is an very useful guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation. The guide is accompanied by a handbook of data collection tools (pdf). They are both very interesting documents if you are interested in evaluating advocacy campaigns.
Glenn
Weighted checklists as an evaluation tool
As I’ve written about previously, checklist are an often overlooked and underrated tool useful for evaluation projects.
In this post, Rick Davies explains about the “weighted checklist” and how they can be used in evaluation. By being “weighted” it means that each item or attribute of a checklist is given an importance – more, same or less compared to other items – and it all tallies up in the overall assessment (see an example here).
New guide to evaluating communications for non-profits and foundations
The Communications Network has just published a new guide “Are we there yet? A Communications Evaluation Guide (pdf) “. The guide has been written for communicators working in non-profit organisations and foundations – it contains interesting case studies and useful advice. Download the guide here (pdf)>>
Communications evaluation – 2009 trends
Last week I gave a presentation on evaluation for communicators (pdf) at the International Federation of Red Cross and Red Crescent Societies. A communicator asked me what trends had I seen in communications evaluation, particularly relevant to the non-profit sector. This got me thinking and here are some of the trends I have seen in 2008 that I believe are an indication of some directions in 2009:
Measuring web & social media: as websites and social media increasingly grow in importance for communication programmes, so to is the necessity to have the capacity to measure what their impact is. Web analytics has grown in importance as will the ability to measure social media.
Media monitoring not the be-all and end-all: after many years of organisations only focusing on media monitoring as the means of measuring communications, there is finally some realisation that media monitoring is an interesting gauge of visibility but not more. Organisations are now interested more and more in having some qualitative analysis of data collected (such as looking at how influential the media are, the tone and the importance).
Use of non-intrusive or natural data: organisations are also now considering “non-intrusive” or “natural” data – information that already exists – e.g. blog / video posts, customer comments, attendance records, conference papers, etc. As I’ve written about before, this data is underated by evaluators as everyone rushes to survey and interview people.
Belated arrival of results-based management: Despite existing for over 50 years, results-based management or management by objectives is just arriving in many organsations. What does this mean for communicators? It means that at the minimum they have to set measurable objectives for their activities – which is starting to happen. They have no more excuses(pdf) for not evaluating!
Glenn
And more on measurement of social media
Still a very new field, there is more and more being done in the field of measuring social media (blogs, wikis, podcasts, etc.). I’ve recently read two interesting opinions on this subject.
First, here is an interesting post from Tom Watson of the Dummyspit blog writing from an IPRA conference in Beijing:
“Measurement of social media was one of the main discussion points. Don Stacks said that data on social media output, such as traffic and click-throughs, was easy to collect but the penetration of messages was much more difficult and mathematically complex. His view was that social network analysis was the way forward with methods, such as Marcovian chain analysis, coming from sociological research.
To understand how messages are being processed and passed along in social media, we need to track bloggers, code the content of their text and work out who is talking to whom.”
Secondly, here is an interesting article from the New Communications Review which speaks of a study to determine “influence and social media” which amongst its findings included:
“For online communities and social networks, the top three criteria for evaluating influence do reflect the importance of online engagement:
o Participation level
o Frequency of posting by the community member
o Name recognition of the individual ”
No doubt, more to come in this area…
Glenn
Key performance indicators for non-profit websites
i’m just back from my first Web Analytics Wednesday (that’s their logo above), held here in Lausanne, Switzerland. I’m interested in web analytics (as I’ve written about before) as it is can be a key component in measuring communications activities today.
The focus of this get-together was on Key Performance Indicators – and in particular KPI for non-profit websites. Here are some of the KPI suggested:
– Bounce rate – number of persons visiting only one page compared to number of people visiting more than one page or vice-versa
– Length of visit (time) – compared to same time last year/month/week
– Depth of visit (number of pages) – compared to same time last year/month/week
And quite some interesting KPI for search:
– Number of visitors using search
– Average number of searches per visit
– % of zero search results (my favorite -high % means people don’t find what they are looking for!)
These are all interesting IKP to think about in monitoring website usage – it also is a good addition to the standard measures usually looked at (e.g. number of visitors and page views).
As we discussed during the get-together, some of these IKP need to be taken in the particular context. Take for example “depth of visit” (average number of pages viewed per visitor). A lot of pages viewed can be both positive and negative. It can mean that someone is really doing some in-depth browsing – or it can mean that someone doesn’t find what they want and is clicking everywhere on the site. A solution was suggested by the WAW moderator, Jmarc Vandenabeele – combine both depth and length (time) of visits. If you have short visits with a lot of pages viewed it could be negative (sign that visitors are clicking on many pages in a short time to find something) whereas long visits with a lot of pages viewed could be more positive.
Glenn
intelligent measurement 