Lies, Damned Lies and Statistics

October 22, 2009

A couple of months ago, the guys over at Socialnomics were talking about a TV campaign developed by an airplane manufacturer in the States, and pointed out that TV might not have been the best medium for this, given the very few people involved in purchasing airplanes. It came up again in conversation recently, and it got me thinking about what might have driven this choice, and the way we use numbers when doing our jobs.

Statistics play a huge role for us, but they can be misleading, particularly when people aren’t used to interpreting them (I’m reminded of this every time I hear a news headline saying ‘prices have fallen this quarter; the rate of inflation has dropped by 2%’).

Saying, for example, that “If you go on TV, ‘100% of your audience will see this ad” is all well and good, but what if your audience is 5 people? Suddenly a TV ad doesn’t seem the right way to go. If you really want them to see your 30” spot, go into their office with a projector – it’s cheaper, it adds a personal touch, and it allows for two- way communication to boot. If readership of a newspaper doubles in a year, that could simply mean that now eight people are reading it, instead of four. That’s not a success story.

Statistics mean nothing without context; every time we take in a statistic, we have to ask ourselves: Where did it come from? Who generated it? Are they independent or are there vested interests? What question did they ask and of whom did they ask it? How big was the sample? There are some statistics we believe without question, but there are plenty we shouldn’t.