Likert scales, frequency and Woody Allen

March 11, 2008 at 9:36 am 3 comments

We often see survey questions with likert scales using frequency estimations such as “sometimes”, “often”, “always”, “never”. etc. However, these scales often provide inaccurate responses as I’ve written about before. Why is that so? Well, describing frequency differs enormously from person to person. This dialogue of a couple seeing separate analysts about their relationship, taken from the Woody Allen film “Annie Hall” illustrates this point:

Woody’s analyst: How often do you sleep together?
Woody Allen: Hardly ever! Maybe three times a week
Diane’s analyst: Do you have sex often?
Diane Keaton: Constantly! I’d say three times a week

So “three times a week” for one person is “constantly” and “hardly ever” for another! This funny dialogue illustrates the point that descriptive terms (e.g. “constantly”) are not accurate measures of frequency.

In survey questions, one way to avoid this is by not using descriptive terms and asking people directly to provide a numerical estimate, for example:

Inaccurate: How often do you watch TV?
Never, hardly ever, sometimes, constantly

Accurate: How many hours per week do you watch TV?
None, under 5, 6-10, 11-15, 16-20, more than 20

Of course, such numeric scales also have problems of accuracy – e.g. people can recall incorrectly how many hours they have spent watching TV. In this regard, it is better to focus on shorter time periods (“what have you done in the past week” or “what do you do on an average day”).

Glenn

Entry filed under: Evaluation tools (surveys, interviews..).

Measurement and NGOs – contradicting voices Found verses manufactured data

3 Comments Add your own

  • 1. khaled Hamuda  |  January 27, 2010 at 9:49 am

    there is no comments, but could you please send to me the names of books or any researches which used or mentioned to the likert scale of six points.

    many thanks for your help

    Reply
  • 2. jb  |  September 26, 2010 at 2:32 am

    Equal-interval and linearity is an issue with any verbal scale. IRT is an effective way to get around this limitation by mapping the response options to a linear scale.

    Reply
  • […] I know that measuring frequency on the Likert scale must be done very carefully. But right now I cannot imagine having numeric choices – once a week, once in a lifetime, […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,105 other followers

Categories

Feeds


%d bloggers like this: