Galluping On with Sloppy Question
Wording ~ #51

Frank Newport, editor in chief of the Gallup Poll and (like
David Moore, featured in Column #50) a vice president of the Gallup
organization, has conducted surveys with multiple deficiencies in question
wording that produce misleading implications. Q1 is the first example

Q1. Just your opinion, do you think that
Charles Darwin's theory of evolution is [ROTATED: scientific theory that
has been well-supported by evidence, (or) just one of many theories and one that
has not been supported by evidence], or don't you know enough about it to say?]

Supported by
Evidence Don't Know No
Yes No Enough to Say
Opinion

2004 Nov. 7-10 35%
35% 29% 1%
2001 Feb. 19-21 35% 39%
25% 1%

"Just your opinion" that starts off the question,
and the alternative choice that ends it, "or don't you know enough to say",
both disempower significant groups of respondents. People who feel that they
are knowledgeable and want their responses respected hear "Just your opinion"
as something else, that Frank Newport doesn't expect them to be able to give a
knowledgeable response. Some may disconnect (i.e., hang-up). Mostly they then
don't care much for what follows and don't bother to think about what they would
otherwise think more carefully about, and finally some may just choose, "Don't
know enough to say". When a somewhat similar large group also hears, "don't
you know enough to say" at the end, they too opt for that choice because
they are intimidated and disempowered.

This phenomenon of surveys have been investigated at great
length (See, Locating Consensus for Democracy – a Ten-Year U.S. Experiment,
p.48 or Chapter 7, "Down Boy”, pp.61-68 of Spot the Spin, The Fun Way to Keep
Democracy Alive and Elections Honest.)Tests of questions with and
without the introductory phrase and other tests with and without the ending
phrase make a big difference in responses.

Some of this is obvious on its face. If the "Don't know
enough to say" option were not offered in Q1, such responses in Q1 would
have been zero instead of 29% in 2004 and 25% in 2001. There are some minor
exceptions here. If "volunteered" responses were allowed and recorded, these
responses would go in what I call the DK bin. That takes care of both choices,
"Don't know enough to say," and "Don't know, for whatever reason".
The latter in Q1 is only 1%, both times, when it was asked in 2004 and in 2001.
The only significant change in responses in almost four years between askings is
a four point increase in "Don't know enough to say". This suggests that
the groups that want to think for themselves and not be intimidated or
disempowered, have been growing.

There is another serious deficiency in Q1. The confusion
of what Darwin's theory is and what it means has evolved in a unique way over
150 years. There are at least two prominent and widely known understandings of
Darwin's theory. No survey question can produce a very useful result without
clarifying that distinction.

Darwin's studies and books made biology a science,
introduced the evolution of species as a concept with broad implications, and
produced a new understanding of the role of humans among all other life forms.
Darwin's concept of "survival of the fittest" has been widely interpreted in
plain language several ways. One is "dog eat dog". Another is species below on
the food chain are eaten by those above and man is at the top of the food
chain. This leads to the belief that the tough, the strong, and the ruthless
survive.

That view overlooks much of reality. Those who survive and
live to produce surviving descendents are often the fittest in an entirely
different way: attracting mates, caring for children, providing for their
families, and generally helping (and being helped by) others.

Victorian English elites, needing to control the masses,
embraced the brute force view of "survival of the fittest", coining the term
"Social Darwinism." Darwin scholars are currently re-appraising this distortion
and suggest that Darwin also believed that humanity's success was due to an
ability to bond, trust, cooperate, and behave altruistically (see
www.thedarwinproject.com).

Between, the two meanings of "Survival of the Fittest" and
other aspects of Darwin's seminal work, especially the origin of species and the
hundreds of magnificent developments in biology based on Darwin's observations
of evolution, any question using Darwin's name needs more context to make the
question clear and make significant respondent groups comfortable with answering
that question.

Another question, Q2, was asked by Gallup VP Frank Newport,
is related to Q1 by subject matter and further illustrates these points:

Q2. Which of the following statements comes closest to
your views on the origin and development of human beings – [ROTATE 1-3/3-1: (1)
Human Beings have developed over millions of years from less advanced forms of
life, but God guided this process, (2) Human beings have developed over millions
of years from less advanced forms of life, but God had no part in this process,
(3) God created human beings pretty much in their form at one time within the
last 10,000 years or so]? Results in the following table:

Man developed over millions of
years God created man Other,
with God but God had no part in present
form No
guiding in the
process Opinion

Even those who are complete believers in evolution might
disagree with the characterization of the evolution of human beings from other
species as described in both the first and second options and might refuse to
accept either choice, perhaps discontinuing or choosing DK (here called by
Newport "Other, No Opinion".) DKs in Q2 here are not negligible, ranging from
4% to 9% (as compared to 1% in Q1). Others believing in million-year evolution
might have failed to accept either choice because neither "God guided the
process" nor "God had no part in the process" reflected their beliefs on how God
manifests Himself.

Conversely many in the plurality that opted for the third
choice might believe somewhat in evolution while also believing in the Christian
God. For example, such people could imagine that some or all early humans were
in a sense "created" less than 10,000 years ago when in various ways some were
changed by God more-or-less as described by the Bible.

In good question design, there need to be alternative choices that together cover all the choices
people desire and without significant overlap. Ideally every respondent finds a
box, just one box and no others, that the respondent is comfortable jumping
into. If 95%+ of respondents did that, that would be good enough. The ideal is
not required.