Cause and effect – goal?
October 23, 2007 at 7:34 am 2 comments
I recently came across a short article in a London newspaper which I summarise as following:
On Friday 21 September, Jose Mourinho, coach of the UK Chelsea football team resigned. On this day, a jump was seen in the number of people visiting a certain job vacancies website. It was proposed by the website marketing team that this jump was due to people learning of Mr Mourinho’s resignation and prompting them to think about changing careers and looking for a new opportunity – thus the rise in number of visits to the job vacancies website.
Doesn’t that strike you as a slightly spurious claim of cause and effect? They appear to be connecting the unrelated and proposing a series of changes that seem slightly unlikely: 1) a change to knowledge (knowing that Chelsea’s coach resigns) to 2) change to attitude (I should change jobs) to 3) behaviour (I actively look for a new job). Possibly of even more concern is that there are no alternative explanations offered as to the jump in website visitors. We can imagine several alternative explanations:
– On Fridays there is always a jump in website visitors (like patterns often seen in purchasing or visits to museums).
– A publicity campaign, a change is site referencing or link campaign drove more visitors to the website on that day.
– Or the more mundane, a technician installed a new monitoring software on the website that led to a more accurate or inaccurate reporting of website visitors.
In addition, no comparison was given to traffic on other job vacancies websites or even global statistics of web traffic (perhaps all sites experienced a jump on 21 September?)
Thinking it through, I think it’s safe to say that there is not enough evidence to point to an association between the resignation of Chelsea’s coach and the jump in visitors to a job vacancies website. The cause (resignation) does seem very distant from the effect (people seek to change jobs). We can point to other cases where an association does seem to exist between a public event and a particular change in behaviour as I’ve written about before: the effect of a reality TV program on the number of people willing to donate an organ and the effect of Kylie Minogue’s treatment for breast cancer on the number of young women taking a scan appointment. In both cases, an association between the cause and effect could be demonstrated and alternative causes were ruled out.
Glenn
Entry filed under: Evaluation methodology, PR evaluation.
intelligent measurement 
1.
Caroline Wilson | October 29, 2007 at 6:11 pm
Good post. I’m fairly confident you’re right and this ‘study’ owed more to creative piggy-backing than it did to rigorous survey technique.
But I guess that so long as everyone’s a winner, no one is going to mind too much – short term anyhow.
The agency gets some useful coverage, and the newspaper gets a topical and football orientated line on an otherwise dull story.
The only potential term losers are the public who’re fed this stuff. However, I think most people can spot this type of hype now so is anyone hurt?
You might be worried that in the long term, the public will no longer believe either PR or journalists.
Or are we too late with that worry ?
2.
Glenn | October 29, 2007 at 9:09 pm
Thanks Caroline, I believe yes, it certain is rather harmless and certainly met the goals of the PR person who dreamt it up – it got a mention in a London daily newspaper. But I think it’s a little like telemarketers who ring you up and say they are doing “social research” when actually they are trying to sell you something – it does a disservice to the name “research”. Just as this does a disservice to the theory of “cause and effect”. And as you say, unfortunately people will believe it but more so it could reflect on the reputation – or reliability – of press and PR.
Glenn