Scrupulous Scribbling for a Saner Society

Survey is no replacement for a proper census

Greater Moncton’s labour force is alive and well and kicking, which may be one reason why fully a-fifth of New Brunswick’s workers will never achieve “Freedom 55”.

Here are a few other “facts” about Canada, now celebrating its 146 birthday, we may not have appreciated, courtesy of Statistics Canada’s National Household Survey 2011, as faithfully reported in The Vancouver Sun:

“We love driving our cars to work. In 2011, almost 93 per cent of people in the workforce drove to work and most drove by themselves. . .English dominates Canadian workplaces, with 85 per cent of the population using the language at work, compared to just 25 per cent who say they use French. . .Chinese languages are the most-commonly used languages in the workplace after English and French. . .Possessing a post-secondary education increases Canadians’ chances of being employed. But there’s no obvious employment benefit to a graduate degree. . .Canadians ages 25 to 34 are far more likely to have trained to be a cook than an auto mechanic or construction worker, when compared to workers closer to retirement.”

Now we know all we need ever comprehend about our families, friends, neighbours; about ourselves in this great and peaceable land. Or do we?

In 2010, back when the federal government announced it was scrapping the mandatory long-form census in favour of a “voluntary” household survey, editorials in just about every major newspaper in Canada screamed their disapproval. The nation’s two top numbers-crunchers Munir Sheikh and Phil Cross actually resigned their posts at Stats Can in evident, if dignified, protest.

In a news advisory at the time, Mr. Sheikh wrote that while he could not “reveal and comment on (the) advice” he gave the government “because this information is protected under the law,” he wanted to “take this opportunity to comment on a technical statistical issue which has become the subject of media discussion. This relates to the question of whether a voluntary survey can become a substitute for a mandatory census. . .It can not.”

Only a month ago, Robert Gerst, a partner in charge of operational excellence and research and statistical methods at Calgary-based Converge Consulting Group Inc., declared in an opinion piece for the Waterloo Region Record, “Take the first data releases from the national household survey of Statistics Canada. . . .The quality of the results has come under criticism because the voluntary survey replaced the compulsory long-form census questionnaire. In effect, this replaced a random sample with a non-random sample. Non-random samples have their place, but making conclusions about the population isn’t one of them.

Naturally, then, “no conclusions about the Canadian population can be drawn from the national household survey. Since making these types of conclusions is the whole point of a census, the survey data is worthless. (This is also true for any survey where participation is voluntary, including citizen, customer and employee satisfaction surveys).”

What, then, justifies the heavy media coverage of the survey? (The Globe and Mail devoted its entire Folio section to the data). Aren’t we, in the Fourth Estate, supposed to suspect this sort of information. In fact, isn’t that our job description? On the other hand, when faced with a bunch of lemons, there’s really only one thing to do.

Intuitively, the survey results seem valid. They track neatly with the more rigorous findings of the 2006 census: Canada’s population was getting older; most commuters in major cities did tend to drive themselves to work; possessing an undergraduate degree did improve an individual’s likelihood of landing a good job. That much appears not to have changed.

But the farther out we go from a proper census (broad, random sampling), the less verisimilitude the findings display. A decade from now, we may no longer be able to trust that the survey results are grounded in enough truth to say anything accurate or useful about our families, friends, neighbours; about ourselves.

In an age when information, increasingly, unlocks the door to economic growth, this seems oddly and regrettably retrograde.