evaluating events and conferences
I’ve written previously about evaluating events and conferences, and in a recent evaluation I undertook for the LIFT09 conference, apart from measurements of attitudes and reactions to the conference, we also looked at what online visibility the conference generated. We found three interesting results based on qualitative and quantiative analysis of blogs and tweets:
– 22% of the blog posts analysed had embedded videos of conference presentations (or liked to them). This is an indication of the importance of the videos in promoting the conference and its themes.
– 32% of the people blogging on the conference were not actually attending the conference – indicating the “reach” of the conference outside of the direct participants.
– The number of tweets on the conference peaked sharply during the three days of the conference (on the second day notably) while blog posts, in smaller numbers, continued to be written about the conference weeks later. The graph below illustrates this point:
Graph data was generated by http://www.technorati.com and http://www.hashtags.org.
The full conference evaluation report can be viewed here (pdf)>>
Global survey on communications evaluation
If you are a communications professional, please take a few minutes and participate in a global benchmarking survey designed to measure best practices in public relations measurement and management.
This survey builds on research undertaken five years ago. The results of the current survey will be presented at the First European Summit on Measurement, June 10-12 in Berlin, Germany.
The Intelligent Measurement blog will also publish a summary of the results once available!
10 tips for translating PR results into the language of business
Dow Jones Insight have produced an interesting e-book “Talk to me – 10 tips for translating the PR results into the language of business“. The e-book is well worth a read, following are some highlights I appreciated:
- 60% of companies are measuring PR/communications at the request of senior management. So as a communication professional it’s probably wise to take an initiative and start measuring activities before management asks for it.
- Measuring just one form of PR/communications activity, such as print media mentions, doesn’t offer a comprehensive view of your strategy – a combination of data is recommended to provide a complete picture. In a recent article by Drs Gregory and Watson, they point towards the trend of using multiple formal and informal metrics and feedback to measure PR/communications.
- As PR/communications is primarily concerned with managing perception and building strategic relationships, one key metric should be measuring relationships. How can this be done? A good starting point is to read the Dr Grunig / Dr Hon “Guidelines for measuring relationships in PR“.
- Presentation of the measurement report is important. Top executves only need a high-level summary of results. Dr Macnamara calls this “two-tier evaluation” explained well in this article (pdf).
US summer program on evaluation
If you are not able to go to Turin and if you are based in the US or close by, you might want to consider the summer evaluation courses taking place in Atlanta, Georgia, USA (pictured above) from 14-17 June 2009.
Organised by the American Evaluation Association and the Centers for Disease Control and Prevention, there are more than 40 workshops on offer over four days, in addition to keynote speakers. Here are some workshops that I would love to attend:
- What Counts as Credible Evidence in Contemporary Evaluation
- How Policy Is Made and How Evaluators Can Affect It
- Making Your Final Report Useful and Used
- Using Social Network Analysis in Program Evaluation
- Taking it Global: Tips for International Evaluation
- Every Picture Tells a Story: Flow Charts, LogFrames
- Evaluation 2.0: Measuring Connected Communications
Summer School in Italy on Monitoring and Evaluation
What could be better than spending some summer weeks in Bologna, Italy (pictured above) and learning about monitoring and evaluation?
Well, the University of Bologna in Italy is organising a summer school on monitoring and evaluation of international programmes and public policies from June 8 – 13 2009.
Further information on the programme:
“The 2009 Summer School On Monitoring And Evaluation Of International Programmes And Public Policies is part of the University of Bologna International Summer School Programme. It is organised by the Centre for International Development (www.cid-bo.org) in cooperation with the Faculty of Political Sciences, the Department of Economics, and the Department of Political Science of the University of Bologna and with the Office for International Programmes of the State University of New York at Albany.
Enrolment fee is € 1,500, and scholarships are available for participants from low-income countries.”
11 hints for a successful evaluation
What makes a successful evaluation? I don’t mean a *success* in terms of producing *pleasing* results – but more so an evaluation that is successfully managed, respects all stakeholders and is of practical use to end users…I’ve been giving it some thought and I’ve come up with 11 hints for a successful evaluation, summarised in below and available with explanatory texts as a fact sheet (pdf):
- Make clear the unclear from the start
- Learn about what you are evaluating from the start
- Invest in planning
- Verify verify verify
- Communicate communicate communicate
- Collect only information you need
- Expect the unexpected
- Report your findings in a brief illustrative way
- Don’t hide the limits of your evaluation
- Don’t be disappointed if nothing happens immediately with your findings
- Give something back for all that you are taking
View the fact sheet with explanatory texts (pdf)>>
Do you have any other hints to add?
What to be avoided when writing evaluation reports
I’ve written previously about what is recommend in putting together a *good* evaluation report.
I came across an interesting fact sheet from the Bruner Foundation on “Using evaluation findings (pdf)”. On page three the authors list eight points to be avoided in writing evaluation reports, sumarised as follows:
1. Avoid including response rates and problems with your methodology as part of your findings.
2. Avoid reporting both numbers and percents unless one is needed to make the other clear.
3. Avoid listing in a sentence or a table, all of the response choices for every question on a survey or record review protocol.
4. Avoid reporting your results with excessive precision.
5. Avoid feeling compelled to keep your results in the same order as they appeared on the survey or the interview protocol.
6. Avoid compartmentalizing your results.
7. Avoid feeling compelled to use all of the information you collected.
8. Avoid including any action steps or conclusions that are not clearly developed from your findings.
Evaluation Professional Development Course on the “Concept Mapping” Method – 18 & 19 June 2009
For those in Switzerland, there is an interesting course (in English) coming up on “Concept mapping” organised by the Swiss Evaluation Society, details as follows:
The concept Systems method guides action plan development, strategic planning, implementation, evaluation and other activities in which group input is valued. Concept mapping takes the ideas of individuals and combines them to form unique visual representations called Concept Maps.
Dates: 18 & 19 June 2009
Location: University of Fribourg, Switzerland
Facilitators: Prof. Bill Trochim, Cornell University and Mary Kane, President of Concept Systems USA
Cost: CHF 700 (reduction of CHF 50 for Swiss Evaluation Society members
Join the communications evaluation group on LinkedIn
If you are a member of LinkedIn, you might want to join the group I’ve created for all people interested in evaluation of communications, public relations programmes, campaigns and projects – wherever you are, wherever you work!
Glenn
Eleven evaluation blogs
The American Evaluation Association has published a list of 11 blogs that focus on evaluation – we are happy to see that Intelligent Measurement blog is amongst them – along with many other interesting blogs. Certainly more evaluators could/should be blogging?
intelligent measurement 