The Power and Pain of Polling

In North America we poll on almost anything we can imagine. What do we think is the softest toilet paper? What’s our favourite soft drink? How do we like our spaghetti? Who do we think we’ll vote for in the next election? We would all like to believe that polling results are either completely useless or absolutely insightful. Sometimes they are both. After all, an insight into how I feel about something today is not necessarily an indicator of how I will behave tomorrow. So we poll again tomorrow to see if the answer has changed and if it has, we poll again in an attempt to determine what influenced that change.

The concept of public opinion as we understand it was first coined by French Foreign Affairs Secretary, Jean-Jacques Rousseau in 1774 and American newspapers started using polling techniques in the 1820s. However, polling in Canada didn’t really take root until the 1960’s. In 1959 the Quebec Liberal Party first used a poll to determine their election strategy for the 1960 election; by 1965 Canadian newspapers were well engaged in the practice of polling. Today, newspapers and political parties are strong adherents to the power of polling and most polling companies have an affiliation with either a newspaper or a political party. There is of course polling related to commercial ventures and while these are lucrative activities and therefore dominate the polling landscape, political polling remains an active portion of all polls conducted.

Despite the popularity of polling there are always questions around the accuracy of polls. Methodology is often called into question if companies conducting similar polls get significantly different results. These differences are a reflection of the quality, tone and tenure of the questions asked. The answers are also influenced by who was asked. After all, if you were to poll a riding that has strong conservative roots, you are unlikely to obtain results that support Liberals and visa versa.

In May 2013, folks in British Columbia were surprised to discover that they still had a Liberal government after the votes were counted in their provincial election. Their surprise and that of the pundits and pollsters came because polls had been clear that the New Democratic party was well in the lead.

“I haven’t trusted polls since I read that 62% of women had affairs during their lunch hour. I’ve never met a woman in my life who would give up lunch for sex.”

One of the most popular approaches or survey techniques is to randomly select individuals and ask them a series of carefully crafted questions. The questions are generally created to avoid bias in the answer. The responses are then tabulated and reported on. While there are a number of ways of gathering information, the thing to consider when you look at polls is that no amount of data, regardless of how accurately gathered will be worthwhile unless you can interpret results accurately. In effect, if the act of polling is a science, then the analysis of polls is an art. The ability to not only interpret but achieve insight into how respondents are likely to react is what makes good polling companies worth their weight in gold.

If you are unclear about what I mean, consider the difference between two questions:

Do most Canadians want their elected officials to behave in a decorous fashion?

Do most Canadians expect their elected officials to behave in a decorous fashion?

Although the questions appear to be almost identical, they will likely elicit completely different responses. What Canadian’s want and what they expect to get can be completely different things. Do you want your kids to listen to everything you say? Do you expect them to listen to everything you say? Understanding these distinctions is what real pollsters bring to the table.

Even when conducting an apparently random poll, pollsters must consider who is actually answering questions. For the most part, many of the polls conducted today are based on random digit dialing, this means you are unlikely to participate in a poll unless you use a land-line. You may correctly assume that people who use a land-line include the majority of citizens in North America, but the question is, are those who rely solely on mobile phones a homogenous group, do they represent a distinct portion of the population? As it happens, they do. The dominant characteristics of those who use a mobile phone almost exclusively, include being young, single and not a home owner. This group is estimated to make up about 7% to 9% of the US population. While this may seem insignificant, remember that these folks are the same people who don’t vote and cannot relate to the issues that most politicians see as priorities. What happens when a generation of people who don’t vote and whose opinions we can’t predict through traditional channels grow up?

What do you think about polling? Do you pay attention to polls? Do you participate in them?

0 responses to “The Power and Pain of Polling”

Don’t have much faith in polling. People will not say anything that could look bad for them or be controversial for that matter. Besides as you point out, the questions are construed so that the pollsters get the answers they want.

Right wing parties are on the rise all over Europe due to the economic depression and massive immigration, mainly from Iraq and now Syria. What middle class person would tell a pollster he/she planned to vote for one of those anti immigration political parties? Not least since you have governments everywhere listening to all telephone conversations.

Catarina you get to the heart of the challenge. Polls are only as accurate as the respondents allow them to be and even then they have fatal flaws. Even if respondents are absolutely honest in their opinion, it doesn’t mean what they will act on that opinion. Economic realities, morality and personal discipline all play important roles.

Mark Twain’s essay “Corn-Pone Opinions” ends up likening public opinion to the voice of God, and in the essay, he drives home the point that there are no original thoughts, just people vying to act like all the other “normal” people. I’ve taken a few polls here and there solely for entertainment purposes depending on my mood. I remember taking one about some Tom Cruise movie and another about Wal-Mart’s service. Did the info gleaned from me matter one iota, maybe… I’m glad there are people who know how to make sense of such data because it’s certainly not my forte.

Jeri I love the use of Mark Twain in this context. Like the voice of God, polls are utterly and completely open to interpretation. The volume and diversity of the things we poll on serves to diminish the relevance of all polls. I remember being polled for waaaaaaay too long on the phone about the toilet paper I preferred. I’ve hadn’t given it much thought up until that point and never will again. 🙂

I follow polls the same way I follow statistics. More often than not those who are paying will skew the information to validate their points. The sad thing is, too many seem to believe in the sanctity of the poll (and statistics). In their misguided view the information is true and unbiased gospel. There is a case for healthy skepticism about everything. It is said to never look a gift horse in the mouth, but then didn’t the trojans forget to do look closely at their gift horse?

Jon, I remember studying statistics in school and coming to the realization that while they could be the factual gathering of opinion/fact/evidence, more often than not the bias of the researcher influenced the questions and therefore outcomes. Even when researchers did their best to be neutral their perspective quietly crept into the way questions were posed. The more important the study, the more critical it was to examine the whole horse.:)

I agree, poll me one day I say one thing, poll me another and my answer will be completely different. I am not opposed to being polled but they have the most uncanny ability to call me at the very worst time possible. Do I pay attention to polls? Sometimes, generally after the fact to see how accurate (or not) they were… LOL.

Susan, I’m convinced that if pollsters have determined one thing with absolutely clarity, it’s when I’m having dinner. Whether we’re catching an early supper or a late night feast, we can be sure that if the phone is ringing while we’re eating, it’s someone doing a poll or selling us something. 🙂

I must be in that 7 to 9% because I don’t have a landline anymore. My family who do, talk about all the calls they get. I haven’t had one in years, and I’m ok with that. The pollsters, however, might be losing out on good data (if that is possible) by not including cell phones in the mix…great discussion!

I envy you the break from the calls. 🙂 Although its not as prevalent in the North America as it is in other parts of the world, I think people without landlines are going to start to become the norm. We spend so much time with our cells, even when we’re at home that it begs the question, why bother with a landline?

I’m not sure if this is still true, but I’ve heard some families will keep landlines for emergency calls, especially when they have children. Others, mostly an older generation, I believe, only use the cell phones when they are away from home. Most of my friends are similar to me in that the cell phone is their primary phone. I hope the pollsters don’t catch up anytime soon!

I went back to look at the demographics for the first half of 2012 on cell phone only use and they are changing fast. I believe that in Canada we have about 1 in 7 households who use just the cellphone and no landline. In the U.S., adults with a mobile phone but no landline is 34%. Of those who use cells, the majority are smartphones. Soon the pollsters will be able to find none of us.

Given how fickle people can be with their answers, and how impatient we all are when asked to do something we did not volunteer for (they called us right) perhaps it is not surprising polls are inaccurate. But then do we really even know what we think? or as Jeri points out, do we just think/answer what we expect people to hear? rather than freak out the poller with a real answer :>

Those are good questions. The purpose of some polls is to draw attention to a product more than actually get an opinion on it. I’ll confess that in order to raise the profile of clients with a targeted audience, I have conducted surveys. I started the survey with the assumption that the client was unknown or poorly known, but by virtue of the survey questions, participants should have a good understanding of the client at the end. In effect, I was asking questions so the participants would learn the answer.

I’m a political junkie (of the US variety), so this was a very interesting post for me. In our last presidential election, the right wing Republicans were convinced that the polls showing that Obama was winning were skewed because of design and polling method flaws and/or being manipulated to give people the impression that he was winning. Needless to say they were shocked on election night when the polls proved to be pretty much spot on. We have lots of polls for presidential politics, including one that aggregates the results and one columnist/blogger for the NY Times (Nate Silver) who is an utter polling wonk and whose explanations and predictions were also pretty much spot on.

I’m a Baby Boomer, so we still have a land line, but I pretty much don’t answer it during election season and AT other times I use callER ID to screen calls. Seriously, if your phone number is identified as belonging to something like “survey USA”, I. AM. NOT. ANSWERING. IT.

When you encounter accurate polls you really have to tip your hat to the pollsters for figuring out how to get to the accurate answers. They have a sea of opposition to get through in order to deliver results. I am a complete political junkie, but my husband is not. Election time at our house is generally made up of me telling him, “be nice to them or at least polite” and him telling me, “tell them not to call here.” It’s pretty funny…to us, probably not so much to the callers.

Debra — I don’t believe that polls have the influence they once had for all the reasons you and the other people commenting state. Just remember the polls had Thomas Dewey beating Harry Truman for the U.S. presidency in 1948. The Chicago Tribune trusted the polls so much that they printed the now famous headline “Dewey Defeats Truman.” Newspapers around the world then used the photo of President Truman, with a wide smile on his face, holding up the paper with the erroneous headline.

The photo of President Truman holding up the newspaper is one of my favorite PR images. It’s a wonderful lesson to all of us in the industry on the dangers of taking our marketing research as gospel. 🙂

I participate in them when I have the time. I often get email requests from companies that I shop with or services I use. If there is no motivation there like free shipping on your next purchase, ect. I don’t often remember to squeeze it in.

Even when I do get freebies I don’t always make the effort. It depends on how often I am contacted and whether I really like the product or service. There are so many ways to determine the preferences of people today, particularly with social media, that I wonder if the polling option won’t eventually start to fade away.

I don’t think polling is accurate but people can be swayed. People will answer the polls based on who is asking for the information. In elections if you were a registered Democrat and you got a call asking how you would vote, and let’s say that you wanted to vote Republican, your “save face” answer may be, Oh I voted Democratic. The other side is, I am not going to let you know who I voted for or who I am considering as it is none of your business. I look at polling like people leaving comments about a product. Half say the product was great and the other half didn’t like it.

So true. The precise wording is so very important in any survey or poll. Therefore, they must be crafted very carefully, and as well, we as respondents must be honest in our answers and be sure we fully understand the intent of the question before we respond.

In paying attention to polls, the important part is: who came up with the poll? What was their agenda? Who are they trying to influence? It’s basically about bias. Yes, I’m cynical when it comes to polling!

I don’t think it’s cynical to question who sits behind a poll or what motivated them to take a poll. The results are presented as facts. Even if the objective of a poll is to raise awareness, you want to know why.

About This Blog

a small shop with big capacity. We help our clients to grow with a blend of communications, public relations and marketing expertise. We add laughter, enthusiasm and dedicated support to deliver awesome results.