Methodology and misuse of research – part 2
July 26, 2006 at 6:42 am 5 comments
As I wrote in a previous post, research results are sometimes misused (that’s nothing new…) and we are often given scant details on how the results were gathered and analysed.
I came across a study undertaken by a bank in Geneva, Switzerland (where I am living) that makes a series of claims about e-banking, web surfing habits and computer use in general. I was surprised to learn that these claims were based on a sample of 300 residents. Now Geneva has some 440,000 residents and I seem to recall from Statistics 101 that 300 people doesn’t really make a representative sample of 440,000 (it would be closer to 600 people depending upon the confidence level and intervals you are aiming at).
I’m not such a stickler on samples given that often the audiences we are looking at can be broken down into sub-populations that are often relatively small in number (so we look for highest participation as possible) – but if you do have a uniform finite population, try using this online sample size calculator to estimate the sample needed, it’s quite useful.
Glenn
Entry filed under: Evaluation methodology, Evaluation tools (surveys, interviews..), PR evaluation, Training evaluation.
intelligent measurement 
1.
Mark McGuinness | August 9, 2006 at 6:07 pm
hi, very interesting blog, this is a hot topic for me as I’m currently using my blog to bring some transparency to my research methodology, by posting podcasts of my interviews as I record them. I’ll leave the recordings up so that anyone who reads the finished report can listen and draw their own conclusions about my interpretation of the data.
2.
Glenn | August 10, 2006 at 1:53 pm
thank you for your comment Mark, as I read on your blog, publishing your questionnaires, source material and podcasts of interviews is a really an important way of showing complete transparency in research. By putting the data online and available, people can see how you have drawn your conclusions – I believe more research should be done this way.
Glenn
3.
Mark McGuinness | August 10, 2006 at 3:22 pm
Thanks Glenn, well it’s an experiment but having started blogging this year, it seemed the obvious thing to do when I thought about the best way to do the research and present the findings. I’ll let you have a copy of the report when it’s done.
4.
Glenn | August 11, 2006 at 6:36 am
Thanks Mark I’m looking forward to reading your conclusions and i’ll keep following your progress (through your rss feed)
Glenn
5.
David Phillips | September 7, 2006 at 3:12 pm
The Hitwise report lists the categories receiving the highest volume of web site visits from 18-34 year olds in the UK.
The only thing I take out of this is that people like searching for content that they want to know about and don’t much like stuff that is shoved at them.
The idea that marketing can somehow meet the value needs and interests of individuals in groups, target markets, stakeholder groups or other ‘demographics’ is showing its age.
The social segmentation, demographic bundle is unravelling.
Bundling up and delivering such groups as market segments can only work in very broad applications. In this case (18-34 year olds) it may not come as a surprise that sex, weddings, social events (where else to find a partner) and testosterone (Tennis and Wrestling?) figure highly.
Not much there for a detailed conversation until there is a closer understanding of individual drivers and where better to find them than in a conversation.