Friday, November 09, 2007

Inform Enlighten Entertain

Thanks to the alert readers who sent in a few recent links from the Calgary Herald that, I think, illustrate some of the problems with journalism in Canada these days.

First up, comes their Wednesday editorial:

For example, we have always suspected the integrity of polls in which newspaper readers tell us they are more interested in stories about equalization than a reported sighting of Brad Pitt at a popular city watering hole. They know the one is important, and should be read; the other actually engages them, and is read -- we suspect -- more widely.

Long time Herald readers will know how the paper tends to shy away from the "issues" whenever possible (Standard Headline: "OMG - Prince Harry!!! Flirting!!!") and I think this is true of most local dailies. However, the Herald has now explained why they do this. In spite of survey and focus group results that say people want hard news and more news, the Herald Editorial board has decided that they know what Calgarians want - stories about Brad Pitt!

Another long time habit of the Herald which I think (hope) hasn't become the norm across Canada is their frequent citing of wikipedia. The latest example of this comes from a story on their favourite topic - Celebrities visiting Calgary!

The Rock and Roll Hall of Fame is in Cleveland, Ohio because of Joel, reports Wikipedia. A total of 15 artists were to be inducted into the Hall of Fame in 1999, which hadn't yet found a home and was going to be built in either San Francisco or Cleveland. Seven artists voted for San Francisco, and seven favoured Cleveland. Thus Joel was asked to make the tie-breaking vote. He chose Cleveland since he considers it one of his favourite cities to play in.

I know it's quite taxing to do proper sourcing but a quick google search of "rock and roll hall of fame" will give you this factoid:

Since its opening in 1995, the Rock and Roll Hall of Fame and Museum has established itself as the preeminent home for the celebration and study of rock and roll music.

I know Billy Joel trivia isn't life or death but the Herald's constant wikiciting is driving me a little up the wall, and it just shows how a little laziness will eventually lead to mistakes.

Remember. Just because you don't want to report real news, it doesn't mean you can't cite real sources.

7 Comments:

Far be it from me to defend shitty journalism, but you have to admit: organizations like the herald have a pretty reliable way of determining what sort of front page stories make people pick up newspapers, and what sort don't.

They count the loonies in their newspaper boxes.

What people say they will do and what people actually do are two totally different things. Whether it's the Herald readers saying they want more real news, or Canadians saying their willing to make personal changes to combat climate change.

Gauntlet makes a fair point. The Herald calls me at home to try to get me to resubscribe about once every quarter. It used to be fun to say "no, you're a shitty paper." Now, they ask (and I paraphrase) "what would make it less shitty?" and, in response to my rough answer of "reporting, and thoughtful consideration of same" respond "did you know we've added x, y and z flavour of the month?" This is depressing because it shows they do their homework and the masses do, in fact, want Brad Pitt.

As long as we are talking about Wikipedia, everyone should check out the article on the word "factoid", because it doesn't actually mean what most of us think it does (nor was it used properly in this post).

Grit: You claim to have attended university, in fact you claim to be taking a Masters in Statistics. Yet you think that the Herald is making shit up when it decides that what people want is different from what they say they want.

Have you ever taken an actual course in statistics? Or is this some arts degree taking a "critical theory" approach to statistics that denies the existence of objective truth, where numbers and evidence are tools of the oppressive patriarchy?

In actual statistics courses, even first year stats, they cover issues of study design and how the responses can be influenced by previous questions, by the questioner, and by the method of evidence gathering. Double blind scientific inquiry is but one of the processes influence by the problems of user response.

In US political polling, Rasmussen (http://www.rasmussenreports.com/public_content/about_us) conducts robo-polls where the surveys are conducted by computer (like voice mail at your favorite customer service line). They have repeatedly proven to be more accurate, as people are more willing to give politically incorrect answers to a non-human investigator.

There is substantial evidence that people give the answers that they know that they should give, rather than what they truly believe. Focus groups are further complicated by internal cognitive delusions: people believe that they want/do things that they know they should. Studies of reported exercise and caloric intake repeatedly show that people overestimate their activity and underestimate their food intake. Biking 3x a week frequently means 1 or two times every two weeks, but really MEANING to bike 3x a week. Calorie intake skips snacks, alcohol (1 glass = 4), underestimates portion sizes, and many other things.

Product failure through focus group is a very common issue. One needs to be very careful in how a group is designed, the method and content of the questions, what the participants know about the goals of the group. People will try to be helpful, giving the answers that they think their questioner wants. Website usability studies commonly have problems with this, and Jakob Nielsen (of Nielsen Norman Group) has a long list of guidelines on improving them, including focusing on what people actually do, rather than what they say they do in response to surveyors questions.

Direct marketers are some of the masters at this, creating cheesy, tacky, unattractive ads that rake money in. High end ad firms notoriously produce excellent ads that people love and appreciate, but don't, er, actually change their behaviour in response.