When journalists encounter statistics, music-downloading edition

The NYT’s Janet Morrissey, this weekend, had a long profile of a music-download service which as far as I can else no one else has even so much as thought about for about three years. The story itself is not particularly noteworthy, but there’s one paragraph near the beginning which encapsulates very neatly a lot of what can go wrong when journalists encounter statistics. Here it is:

About 95 percent of music downloads in 2010 were unlicensed and illegal, with no money flowing back to artists, songwriters or record producers, according to Alex Jacob, a spokesman for the International Federation of the Phonographic Industry. So riches could await a company that persuades some of these Internet scofflaws to change their ways.

This paragraph is structured in a standard journalistic manner: start with a fact, then say what your source was for that fact, and then draw a conclusion from that fact. Except that in this case, the standard journalistic formula is twisted inside out.

First of all, the central assertion is almost certainly wrong. I feel comfortable in saying that it’s simply not true that 95% of music downloads in 2010 were illegal. That number was, literally, incredible in January 2009, when the IFPI first trotted it out as a number averaged over a three-year period. And since then, the iTunes music store and other legal services have grown very fast. What’s more, even the IFPI seems to have backed away from the figure: it appears nowhere in its 2011 Digital Music Report. The highest such number there is the claim that “in the UK, for example, 76 per cent of the music obtained online in 2010 was unlicensed”.

In any case, the last people you’d trust to come up with a reliable figure for such things would be the industry group devoted to demonizing and exaggerating the threat of illegal downloading. Morrissey doesn’t seem to understand, here, that you can’t just quote the IFPI and be done: the fact that the number is coming from the IFPI, far from being a reason to believe it, is a reason not to believe it. At the very least Morrissey should have asked whether the number was based on any kind of empirical research; instead, she felt comfortable simply parroting the nonsense being put out by the paid flack for a pressure group, and reporting it as fact.

And what’s more, the statistic isn’t even relevant. Let’s say I download 361 albums one night over Bittorrent, and I have 19 friends who each download and pay for a single album on iTunes. Then it would be true to say that among me and my friends, 95% of the music downloads were unlicensed and illegal. But it would also be true to say that 95% of us are downloading music legally and paying for it. Would riches await a company that persuaded the scofflaws to change their ways? No, because there’s only one of me. And if I start paying for the music I download, you can be sure I won’t be downloading 361 albums in one night.

Before you start quoting statistics, then, it’s always worth (a) knowing where exactly they come from; (b) verifying them independently if you were fed them by some pressure group; and (c) making sure that they say what you say that they say. Otherwise, you just end up looking credulous and silly.

Update: IFPI’s Alex Jacob responds in the comments, saying that his 95% figure is “based on independent research in a series of markets around the world”.