28 July 2009

There's been strong reaction to radio Triple J's Hottest 100 of All Time, as blogged about here previously.

That reaction hasn't been uniformly critical. In fact, there's been a considerable backlash, much of it based on the erroneous belief that if a poll is large enough - and this one had 500,000 entries - it must be right.

Wrong.

Any poll is only ever as good as the sample, the questions asked and how the results are gathered and analysed.

And the findings of this one don't have what researchers call 'face validity'. The problem isn't with any of the individual songs on the final list - each no doubt has its proponents. It's the big picture statistics that don't lie. When even Triple J announcers and fans are surprised and dismayed that not one of the supposed 100 greatest songs of all time is by a female artist, it suggests some pretty significant errors.

"Errors?!!" I hear you say. "But it can't be wrong - it's a poll. It's about people's opinions, so it must be right!"

People's opinions are never wrong. Absolutely never. But opinion polls often have errors that render their findings wrong. I'm using the word "error" here in the sense used by researchers and statisticians to describe problems in research design, analysis and interpretation. Let me explain.

If we accept that a TRUE Hottest 100 songs of all time exists out there in the minds of Triple J listeners, then the idea of the poll is - within practical limitations - to capture that collective mindset with an acceptable level of accuracy.

"Error" refers to any problem with the methodology that could contribute to the end result of the poll not being a reasonable reflection of what's actually in the collective mindset of our population of interest.

First and foremost is "sampling error". In any voluntary poll, the findings are only representative of those who actually vote. That naturally means that people who are particularly passionate about the cause (in this case, a particular song or artist) will vote. In other words, they aren't representative of the whole population - statisticians call this a biased sample. But people who didn't vote can't complain, as they only have themselves to blame.

In any case, what we have read about the scale of the Triple J poll (some 500,000 votes) and the spread of age and gender means you'd be hard-pressed to blame sampling error for the complete absence of female artists in the Hottest 100. So it's back to the methodology...

The next type of error is to do with the survey itself. We know what we were looking to find, but did we ask the right questions?

Triple J could have asked everyone who voted to nominate his or her top 100 songs in order, and then counted every vote and applied some kind of weighting based on that order.

But that's not what happened. In fact, Triple J asked listeners to nominate only their top 10 greatest songs of all time. You can well appreciate why Triple J would do this for practical reasons, but it introduces some significant sources of error.

Firstly, there's what I will call the Tenacious D effect. The instructions and the task are likely to have suggested to many people (consciously or unconsciously) that if they had to choose the 10 "greatest songs in the world" then these must be truly "awesome" songs.

Not surprisingly, the final list of the Hottest 100 was heavy on anthemic, epic, deep and meaningful power ballads - the kind of things that get played at funerals (yes, even Heath Ledger's). There are very few "feel good" dancefloor-fillers. And it appears to have helped a song considerably if the artist died in tragic circumstances.

Secondly, compiling a Top 100 out of thousands of 10-song samples introduces a very significant statistical problem. What you end up with is a sampling distribution of people's Top 10s, and NOT a true list of the Hottest 100. And that produces very unrealistic results, as per the following example - the figures are made up, but they illustrate the problem.

Let's put the Tenacious D effect aside for now and assume we asked a large group of people to list their Top 100 songs in order.

10% said Aretha Franklin's "Respect" was one of the 100 greatest songs of all time, and 0.5% of people had it in their Top 10

50% of people agreed that Nirvana's "Smells Like Teen Spirit" is one of the 100 greatest songs of all time, and 10% of people had it in their Top 10

Around a third of our sample said they'd never even heard of "Chop Suey" by System of a Down but 5% said it was one of the 100 greatest songs of all time and half of those (2.5%) had it in their Top 10.

So now we compile our list.

If we use all the votes to compile the Top 100, then Nirvana ranks above Aretha, with System of a Down lower down the list.

But when we only count people's Top 10s, Nirvana still - rightly - ranks high in the Hottest 100 and System of a Down makes it in towards the bottom but Aretha Franklin doesn't show up at all.

Without having access to any of the specific figures from the Triple J poll, the number of songs in the final Hottest 100 that could be considered relatively obscure - even for a Triple J audience - strongly suggests that these kind of survey and statistical errors are to blame for the lack of diversity that has bothered so many people.

How can Triple J design a poll that does a better job of finding the TRUE Hottest 100 of all time? Well, talk to some market researchers and statisticians to begin with.

27 July 2009

Every day we read (on the internet, of course) of another old and revered print newspaper in a US city being read the last rites. The old business model just doesn't work any more. Advertising has moved to the web, printing costs are exorbitant, etc. etc. Fair enough.

But you don't have to go far to see some of the negative implications of this. A daily glance at headlines, links and copy in the online versions of many newspapers - including the so-called "quality press" - illustrates how publishing processes and priorities have changed.

It seems likely that news organisations are replacing sub-editors - people who knew how to get words to work - with IT types whose skills lie elsewhere and who are tasked with getting the words they are given up on screen as quickly as possible.

Consider the recent reporting of the Tour de France by Fairfax cycling correspondent Rupert Guinness (pictured above). You may have seen Rupert in a succession of gaudy Hawaiian shirts serving as a guest commentator on SBS Television at the end of several stages of the race.

However, within minutes of the stage finishing, the sparks began to fly between the Columbia team for which Australian riders Michael Rogers and Mark Renshaw are signed with, and the rival Garmin team who has Australian Matt White as one of their sports directors and had one of their riders in the 12-man breakaway.

What the...?

Firstly, when did so-called journalists cease to be able to write grammatical and readable copy? Producing acceptable clean copy used to be one of the basic rules of journalism - if you were a sports type who couldn't write, then you had a ghost writer or a sub-editor to clean up your copy for publication.

Second, how can a reputable "quality" news organisation allow such amateurish material to be published... and, worse, to sit there, uncorrected, more than a week later (as I write this). Clearly no-one literate at Fairfax has actually read the story.

OK, perhaps Rupe was under some pressure to file quickly. But the Besançon stage was over by 2am Sydney time and the byline on the story says 6:28am - surely plenty of time for a professional like Rupert to file something half-decent and enough to allow a sub-editor to make some sense of his mess (and to call/email him back to say "clean up your act").

Grammar and spelling are critical for ease, clarity and accuracy of communication. They DO matter - online as well as in print.

13 July 2009

I compiled this list of 100 great songs from female artists and black artists. All of them appear on multiple online lists of "100 greatest songs..." (of all time, different genres, different decades, etc.). Sources include VH1, Billboard, Mojo magazine, Grammy winners.

It's just a starting point, but I defy anyone NOT to find at least a few songs here worthy of displacing some of the anthemic, epic, angry and emo white boy rock that dominated Triple J's list. (Note: Alphabetical order, not order of merit). Nominations and challenges welcome - it's all in the interests of diversity.

And let's face it - if Elton John's Tiny Dancer could get in, then nothing's off-limits!

The Hottest 100 brand has become extremely important for Triple J since the poll began in the late 1980s. The annual Hottest 100 CD compilations, which began in 1993, sell by the hundreds of thousands. The annual listener poll for the year's best releases and the accompanying CD and events are a central and vital part of the station's promotion and merchandising.

But the results of this year's poll make disturbing reading for Triple J management and its ultimate masters higher up in the ABC and the Federal Government.

The concern won't be about any individual song or artist - whether a particular song is worthy of inclusion in the best 100 of all time will always be subject to personal taste and passing fads, and will be a subject for robust and enjoyable debate.

But it's the overall picture painted by some basic stats from the Hottest 100 that should have Triple J management worried.

Firstly, only five black artists or acts are represented in the entire Top 100: Michael Jackson (whose recent untimely death no doubt gave him a boost in the poll), Jimi Hendrix, Bob Marley, Stevie Wonder and (maybe) English band Bloc Party (fronted by a man born in the UK to Nigerian parents).

It is inconceivable that any reasonable poll of the "Hottest 100 songs of all time" could so glaringly exclude any and all black artists and music since 1982. This is especially disturbing when one considers how culturally important, if not dominant, essentially black musical forms like soul, funk, hip-hop and R&B have been in Western popular music since the 1960s. Triple J (in)famously began its national broadcasts in 1989 by being the only radio station in the world to play the song "F@#* Tha Police" by NWA (Niggaz With Attitude). Yet there is not a single black hip-hop act in this Hottest 100.

However, far more glaring than the relative lack of black artists is the total absence of female artists.

At best, I can see two songs that even feature female voices - Teardrop by Massive Attack (with "vocals by Elizabeth Fraser", i.e. not a full artist credit) and vocals by Kim Deal on the Pixies' Where is My Mind.

Again, it is utterly inconceivable that any list of the 100 greatest songs of all time could totally exclude female artists.

I know Triple J's own playlist and announcers don't reflect this shocking lack of diversity, and I'm not suggesting an editorial hand at work here. But if you do as I did and Google "greatest 100 songs" (of all time, by genre, by decade, etc.), you'll find all kinds of takes on this - polls, subjective lists, critics' choices. Yet you won't find a single list that completely excludes females and all but the most white-acceptable performers of colour. Indeed, from just some of these lists, and in the space of an hour or two, I easily compiled this list of songs by black and female artists.

Some people - including Triple J on-air personnel - have defended the Hottest 100 over the last 24 hours using what I call "the Logies excuse", i.e. that it's a popularity contest and it's not about merit.

Problem is, Triple J has successfully leveraged its listeners' musical tastes for years in the form of the Hottest 100 brand. And half a million votes sounds like great level of audience engagement. But previous annual polls and compilations have never reflected such an overall narrowness of musical style and audience appeal.

The top 20 especially was dominated by anthemic, epic, heavy, "message" songs in minor keys. And if the singer died tragically and prematurely, the song placed even higher. No "feel good" songs. And definitely no dance.

All of this makes this Hottest 100 of All Time a poison chalice for those whose job is to try to promote the station on the back of it, and for those who would defend Triple J's taxpayer funding on the basis that the national youth broadcaster has an important role in promoting cultural diversity.

Unfortunately, the data suggest that Triple J - intended to break down the sameness associated for so long with commercial Top 40 radio - may simply have subsituted a new kind of (white male) sameness.