Writing better survey questions
I’ve written previously about resources for putting together surveys and I’m often asked about the “wrongs and rights” of writing questions for surveys.
So I’ve put together a factsheet (pdf) that has 12 hints on writing better survey questions using real examples – both good and bad.
View factsheet on writing better survey questions (pdf) >>
Glenn
Challenges in measuring social media
Tom Watson of the Dummyspit blog has brought to my attention a new report (pdf) from the Society of New Communication Research that examines the elusive question about how to measure the influence and effectiveness of social media, such as blogs, podcasts, etc.
As the report states although social media is clearly changing “the way we think about media and influence … [companies] are still struggling to find effective metrics for deciding who are the influential players” (p.16).
The report summarises current measurement and evaluation practice as:
- Top criteria for determining the relevance or influence of a blogger or podcaster are quality of content, relevance of content to the company or brand, and search engine rank.
- For evaluating a person’s influence in online communities and social networks, the main measures are participation level, frequency of activity and prominence in the market or community.
- About half the surveyed communicators formally measure their social media activities. Their goals are “to enhance relationships, improve the reputation of their businesses, drive customer awareness of their online activities and solicit customer comments and feedback.” (p.16)
There are certainly some valid points there – read Tom’s full post here, just be aware of some of the more dodgy social media metrics that are out there…
Glenn
New portal on communications evaluation
A new portal has been launched on evaluation communications – with plenty of interesting resources. The Communication Controlling portal is supported and updated by experts from Leipzig University and the “Value Creation through Communication” working group of the German Public Relations Association (Deutsche Public Relations Gesellschaft, DPRG). Visit the portal>>
Automated network mapping
An interesting development in network mapping is the ability to do it automatic and “live”. I’ve been involved in doing network mapping of conferences the old-fashion way – manually (as I’ve written about before). This article in the Technology Review explains how automated network mapping was done at a conference. Simply put, conference participants were given a badge (as pictured above) that tracked their proximity to other badges and sent this data to a central computer which then analysed the data and produced a real time visualisation of the event’s social network (pictured above). Interesting…
Guidelines for Evaluating Nonprofit Communications Efforts
For those interested in the area of campaign/advocacy evaluation, here are some very good guidelines on evaluating nonprofit communications efforts (pdf) from the Communications Consortium Media Center. The guidelines focus on “lessons learnt” in the evaluation of campaigns and sets out some key concepts to consider.
Glenn
What is a survey?
Just came across the “what is survey?” website which is a very complete guide to surveys from the US National Opinion Research Centre. The guide is also available in pdf (pdf version here).
In an earlier post, I also listed other useful guides about surveys.
Glenn
PR Measurement – Catch 22
PR Week has recently published their Marketing Management Survey (pdf). Amongst other subjects, the survey asked US-based marketing excutives about measurement and their activities. Ed Moed of the Measuring Up blog points out that the survey “rehashes many of the same weary and misguided perceptions” on PR measurement, notably:
“PR is a difficult one to measure.”
“ Clients value PR, but question its value because they think it can’t be measured”
“In an economic climate where budgets are tight, research and measurement are very often the first portions of a PR budget to be cut. Yet measurement is necessary to prove ROI, which can help increase budgets, providing a Catch-22 situation…”
As Ed points out “People, public relations can be measured.” I couldn’t agree more, read Ed’s full post on his blog.
Glenn
From broad goals to specific indicators
No doubt you have heard of the Millenium Development Goals (MDGs), eight broad goals on poverty, ill-health, etc, agreed upon by all countries to try and reach by 2015.
From a monitoring and evaluation point-of-view, what is interesting is that these goals are broad sweeping statements, such as:
Goal 1: Eradicate Extreme Hunger and Poverty
Goal 3: Promote Gender Equality and Empower Women
One could ask – how can these broad goals be possibly monitored and evaluated?
As detailed on this MDGs monitoring website, what has been done is to set specific indicators for each goal, for example:
Goal 3: Promote Gender Equality and Empower Women
Description: Eliminate gender disparity in primary and secondary education, preferably by 2005, and in all levels of education no later than 2015
Indicators:
3.1 Ratios of girls to boys in primary, secondary and tertiary education
3.2 Share of women in wage employment in the non-agricultural sector
3.3 Proportion of seats held by women in national parliament
So from broad goals, the MDGs focus on two to seven specific indicators per goal that they are monitoring. That’s an interesting approach, as often we see broad goals set by organisations and then no attempt made to actually detail any indicators.
the MDGs monitoring website plays an active role in monitoring these indicators combining quantitative data (statistics) and qualitative data (case studies) – also an interesting approach to show how such indicators can be tracked.
Glenn
Evaluating communication products
Organizations spend millions on communication products every year. Brochures, annual reports, corporate videos and promotional materials are produced and distributed to a wide variety of audiences as part of broader communication programmes or as “stand alone” products.
However, working with many different types of organizations, I’ve noticed that little systematic follow-up is undertaken to evaluate how these products are used and what is their contribution to achieving communication or organizational goals.
I recently worked on a project where did just that – we evaluated specific communication products and attempted to answer the following questions:
- Is the product considered to be of high quality in terms of design and content?
- Is the product targeted to the right audiences?
- Is the product available, accessible and distributed to the intended target audiences?
- Is the product used in the manner for which it was intended – and for what other unintended purposes?
- What has the product contributed to broader communication and organizational goals?
- What lessons can be learnt for improving future editions of the product and design, distribution and promotion in general?
The results were quite interesting and surprising. We were also able to map out the use of a given product, like in this example:
You can read more about this approach in this fact sheet (pdf) >>
Glenn
Resources for writing surveys
Following are some useful resources for writing surveys:
A brief guide to questionnaire development (pdf) – very good guide with some interesting points on types of questions to use.
A guide to good survey design (pdf): Very comprehensive guide including “pitfalls with questions” (go to page 59).
Online survey design guide: a whole website dedicated to writing better online surveys.
20 tips for writing better surveys – good tips on managing surveys and improving response rates.
And don’t forget my own Ten tips for better web surveys (pdf), Seven tips for better email invitations for surveys, Writing open ended questions and Analyzing open ended questions.
Glenn
intelligent measurement 