Normally
this group plays
in the senior amateur
league, and we would not claim more. Today my
theme is one on which you and I are experts: screwing things up.
Simply by staying alive for 60 years or so gives us ample experience
of our own mistakes, mistakes by those around us, mistakes by the
leaders and electorates of our countries. Last year we saw
the British vote for Brexit and the American one for Donald Trump. We
are aware of Japan's attack on Pearl Harbour, Napoleon's and Hitler's
invasions of Russia, and Chamberlain's trust in Hitler at Munich. The
poppies we wire on
Saturday remind us of
the dreadful mistakes of August 1914, when sensible statesmen who did
not hate each other drifted into a cataclysmic European war. In both small and
large cases we ask:
what were they or we
thinking of? Wasn't it
obvious at the time that these decisions were really dumb?

If
we can improve our chances to avoid errors of judgement only a little
through understanding, the effort is well worth it. This is a
philosophy group, and we look for general rather than particular
insights, and I won't relitigate the examples. Fortunately there has
been some recent and convincing research on systematic and probably
innate biases of human nature that lead us into error. This is worth
knowing.

Our
guide will be cognitive psychologist Daniel Kahneman, and his very
readable book Thinking, Fast and Slow (“TFS”
below) (Ref 6),
published in 2011. In this he summed up his life’s work, most
of it in a close collaboration with Amos Tversky. Their collaboration
was immensely productive, and ranks with that of Crick and Watson in
the cracking in the genetic code. Tversky was apparently the theory
and ideas man, like Crick; Kahneman more the careful workhorse, like
Watson (though both seem much nicer people). Tversky died in 1996.
Six years later, Kahneman received the Nobel Prize in economics –
there isn't one in psychology. If you don’t feel like buying
the book, his Nobel Prize lecture is free (Ref 7).

Tversky
and Kahneman are or were Israeli, and much of their early work was
carried for the Israeli Defence Forces, who were very interested in
the often very costly mistakes made by their pilots and soldiers.
They were much less interested in blaming the young men who made
them, who were often dead anyway. This intensely pragmatic approach
marked the style of the two researchers, and their dismissal of the
moralising lens is refreshing and constructive. Moral condemnation
has its place, but it’s not an explanation.

Before
we get to the recent findings, I would like to look briefly at three
other explanations: machine error, Aristotle’s weakness of will
– emotions overcoming reason - , and Bacon’s idols.
Actually I won’t: you will find my thoughts on these in the
printed version of this paper on my website (Ref 9). For now, let me
just stipulate that Kahneman’s cognitive errors don’t
cover everything and don’t claim to.

I:
Machine error

We
rely heavily on automatic cognitive processes, for example when
walking or cycling or driving a car or in sport. These sometimes go
wrong: we think we can get round the bend at 80km/h when it should be
60, or we trip on our shoelaces. The same goes for our cats: Pussy
may guess wrong which way the mouse will run, or wrongly think she
has time to cross the road in front of a car. In both cases, the
consequences may be anything from trivial to tragic. What they are
not is a puzzle. We face exactly the same problems building a robot
to do the same things. The algorithm may be buggy, the processor too
slow, the sensors inaccurate, the bandwidth of the communication
channels too restricted, or we may be defeated by countermeasures.
The way to reduce such errors is learning and practice. Polar bear
and lion cubs are born with an instinct to hunt, but they are at
first very bad at it, and only acquire skill from their mothers.

The
puzzles are about our conscious bad decisions. Why should we
make them at all? We are descendants of 3 billion years of survivors.
We are the most intelligent animal, with sophisticated sensory and
motor skills and reasoning abilities. When we can't solve a problem
by ourselves, we have speech and social networks so we can bring in
the help of others and the experience and wisdom of a community.

II
: Blame the passions

There
is bad luck of course, but it's an excuse rather than a theory. The
oldest general theory I can find is “reason overcome by the
passions”. It’s implicit in the myth of Cain and Abel.
Aristotle used the term akrasia, usually translated as
“incontinence” (Ref 2). It's also the foundation of his
theory of tragedy (Ref 3) as the destruction of a hero by a tragic
flaw, an uncontrolled desire overwhelming common sense: Oedipus'
obsession with learning his origins, Paris' lust for Helen,
Agamemnon’s desire for glory. We can add Othello's jealousy and
Macbeth's ambition. The Stoics went so far as to call all passions
intrinsically evil. Aquinas is okay with passions only as slaves to
reason (Ref 1).

Aristotle
does distinguish between akrasia
where the emotions
simply sweep aside reason (Othello) and the more common case where
they seem to hijack reason, and we talk ourselves into error with
specious arguments (Macbeth), without any loss of self-control. The
point was made elegantly
by the Oxford
philosopher J.L. Austin, in this bloodcurdling description of
academic depravity1:

I am very partial to ice
cream, and a bombe
is served divided into segments corresponding one to one with the
persons at High Table: I am tempted to help myself to two segments
and do so, thus succumbing to temptation and even conceivably (but
why necessarily?) going against my principles. But do I lose control
of myself? Do I raven, do I snatch the morsels from the dish and wolf
them down, impervious to the consternation of my colleagues? Not a
bit of it. We often succumb to temptation with calm and even with
finesse.

Clearly
this mechanism is real. All too often we sit down to play poker with
men called Doc, pursue women nicknamed Good-time Gertie or men known
as Heartbreak Henry, seek wealth at the expense of happiness, and so
on. There are two difficulties. One is that there are many cases
where it doesn't seem to fit. The Japanese did not attack Pearl
Harbour out of an unreasoning hatred for Americans.

A
bigger one is that the passions – immoderate passions, not
milk-and-water tame ones - drive success as well as failure. Henry
VII was as ambitious as Richard III, and it worked out well for him
and his country. Cortez, Darwin, Gates, Beethoven were ambitious men.
In our own lives, we can very probably think of instances when our
desires have overwhelmed our judgement and led us to small or great
disasters. But when we look on the other scale at our best decisions
and achievements, were these not also driven by the strong desires
that enabled us to take a chance and strive for some difficult goal?
We seek out sexual partners and life companions out of need, and very
often find them. We could not in fact function as passionless
automata. Our desires provide us with the frame of values that allow
us to choose, and the energy to put our choices into effect. So it’s
wrong to see the passions as automatic enemies of good decisions.
There must be something else as well.

III
: Bacon's Idols

Fast-forward
2000 years to the Renaissance. You have heard of Francis Bacon, one
of the founders of the scientific method. He was also a cold
careerist:
he abandoned
his protector Essex just in time, signed
off
on torture warrants as James'
Attorney General, and was
eventually fired for taking unusually
large bribes. Still, of
all the world's great thinkers, he was the one with the longest
career in high-level politics. Bacon left us in
his Novum Organum(Ref
4)a very picturesque
proto-theory
of cognitive mistakes,
which he labeled Idols, objects of false devotion, fixations
in error. He proposed four:

“The
Idols of the Tribe have their foundation in
human nature itself, and in the tribe or race of men. ... The human
understanding is like a false mirror, which, receiving rays
irregularly, distorts and discolors the nature of things by mingling
its own nature with it.” (§41)

Examples
are the tendency of the human understanding “to suppose the
existence of more order and regularity in the world than it finds”
(§45) and
confirmation bias (§46).
Fair enough.

“The
Idols of the Cave are
the idols of the individual man. For everyone (besides the errors
common to human nature in general) has a cave or den of his own,
which refracts and discolors the light of nature, owing either to his
own proper and peculiar nature; or to his education and conversation
with others; or to the reading of books, and the authority of those
whom he esteems and admires; or to the differences of impressions,
accordingly as they take place in a mind preoccupied and predisposed
or in a mind indifferent and settled; or the like.” (§42)
This
fits Trump all right.

“There
are also Idols formed by the intercourse and association of men with
each other, which I call Idols of the Market Place, on
account of the commerce and consort of men there. For it is by
discourse that men associate, and words are imposed according to the
apprehension of the vulgar. … Words plainly force and overrule
the understanding, and throw all into confusion, and lead men away
into numberless empty controversies and idle fancies.” (§43)
Twitter!

“Lastly,
there are Idols which have immigrated into men's minds from the
various dogmas of philosophies, and also from wrong laws of
demonstration. These I call Idols of the Theatre,
because in my judgment all the received systems are but so many stage
plays, representing worlds of their own creation after an unreal and
scenic fashion. Nor is it only of the systems now in vogue, or only
of the ancient sects and philosophies, that I speak; for many more
plays of the same kind may yet be composed and in like artificial
manner set forth.”(§44)Marxism!

This
is clearly very interesting stuff. There are two problems. One is
fair enough: the focus of the Novum Organum is theoretical not
practical belief. He does not draw on his experience of the world,
his close observation of the end of Mary Queen of Scots,, the
executions of Babington, the Jesuits, and Guy Fawkes, the defeat of
the Spanish Armada, and the fall of Essex. This is not the hands-on
Bacon who writes, in the Essays (Ref
5, Of Counsel),
of the importance of seating plans in meetings. It is also
exceedingly sketchy. The main conflicts of belief in his day were
about religion, and being explicit would possibly have been dangerous
and certainly lost him half his potential audience in Europe.

The
bigger problem is that for all his advocacy of the scientific method,
Bacon presents zero evidence for the Idols, not even learned
anecdotes from classical authors. They just serve as bogeymen to
contrast with the true method, the scientific one. The brilliant
speculation sparked no research, and passed into obscurity.

IV
: Psychology meets experiments

Unlike
Bacon’s or Aristotle’s, the work of Kahneman and Tversky
is based on actual experiments, replicated and documented. There are
two sides to it. One is a set of effects. The other is a general
explanation for them. I won't try to cover all the effects, it's more
important for you to get the style and method.

Experiment
1

The
first experiment is a quite famous one about a woman called Linda.
The test subjects were shown a short description and a list of eight
possible outcomes describing her present employment and activities.
I'll leave out the dummy questions and ask you to write down your
answers. Here's the description.

Linda is 31 years old, single,
outspoken and very bright. She majored in

philosophy. As a student she
was deeply concerned with issues of discrimination and social justice
and also participated in antinuclear

demonstrations.

Based
on this information, please rank the following in order of
likelihood:

A. Linda is currently active
in the feminist movement.

B. Linda is currently a bank
teller.

C. Linda is currently a bank
teller and active in the feminist movement.

How
many of you put A first? Everybody.

How
many put C second? A majority. That is absolutely standard. Whoever
you ask, you get 85%-90% saying the same. That includes doctoral
students at Stanford in a programme on decision science. But the
majority is wrong. As a matter of logic, the joint probability
of two outcomes must be lower than that of either. Suppose
we rate the feminist probability at 95% and the bank teller one at
5%. The joint probability
is 4.75%.

What
happened here? The description shouts FEMINIST! at us. The light bulb
is so bright it cuts out logic. This is the fallacy of similarity: we
use a criterion of representativeness to stand in for a judgement of
probability.

You
may say: that was true but artificial, a trick question. So let's try
another which is closer to the problems we face.

Experiment
2

Here
is a profile of Tom, a graduate student, written by a psychologist
when Tom was in high school, on the basis of personality tests of
dubious validity:

Tom W. is
of high intelligence, although lacking in true creativity. He has a
need for order and clarity, and for neat and tidy systems in which
every detail finds its appropriate place. His writing is rather dull
and mechanical, occasionally enlivened by somewhat
corny puns and by flashes of imagination of the sci-fi type. He has a
strong drive for competence. He seems to have little feel and little
sympathy for other people and does not enjoy interacting with
others. Self-centered, he nonetheless has a deep moral sense.

Participants
were given this description along with a list of nine fields of
graduate specialization, including computer and library science, as
well as business administration. They were again asked to rank the
fields in order of likelihood. Computer and library science came out
on top, business administration low down.

There
is no logical
incompatibility here, but
it is still unsound to blindly follow the stereotype.
The problem
is
that there are six
times as many graduate students in business subjects in the USA as in
computer and library science combined2.
If
we know nothing about Tom's character, the best way to assess the
probability is to use the base rate. This means that we
should
put business as six times more likely than computer science. In the
experiment, the subjects were in addition told that the profile was
of dubious validity, so it should have been ignored. Even if it were
accurate, the best way, as recommended by the Reverend Bayes, is to
start with the base rate and adjust for additional information. So we
should start at 6:1 for business and mark down. Doing
it this way, we are not very likely to get to equal odds, let alone a
lead for computer science.

Base
rate errors of this type are very common and surely of practical
importance. Failure to use them properly is compounded by massive
ignorance of what they are. In a survey of students at Eugene
(Oregon) asking them to compare risks of death, the average estimate
of the relative risk of death from accidents
and
from diabetes was 300
to 1 for accidents.
The true
ratio is
1 to 4; diabetes is
by
far the higher risk (TFS,
Ch. 13). One
of the secrets of professional expertise may well be better knowledge
of base rates.

Experiment
3

Another
way we fail at statistics is looking for narratives in the most
picturesque and
salient
rather than the most relevant features of a problem. A survey looked
at death rates from kidney cancer in the 3,141
counties in the United States in a recent year. The lowest
rates were in sparsely
populated, rural
counties in the South,
West
and Midwest, typically with Republican politics. Your
brains are setting to work. Is it the Republican politics? Probably
not. The greasy food, alcohol and
opiates?
Nah. Something about rural.
Maybe it's air pollution.

Now
what if I told you about the counties where deaths from kidney cancer
were highest:
sparsely
populated, rural
counties in the South,
West
and Midwest, typically with Republican politics (TFS,
Ch. 10).
What is going on
here?
The key adjective was the one you ignored. It's all about small.
Small counties will always show more extreme results than big ones,
up and down. Throw two dice, and your chance of both sixes is 1 in
36. Throw four dice, and your chance of all sixes is 1 in 1,296. Bill
Gates has spent a fortune promoting small schools, which have the
best scholastic results. Sure they do: they also have the worst ones.

Experiment
4

That's
enough statistics for now. Let's look at framing, through another of
Kahneman and Tversky's problems.

Imagine that the United States
is preparing for the outbreak of an unusual Asian disease, which is
expected to kill 600 people. Two alternative programmes to combat the
disease have been proposed. The exact scientific estimates of their
consequences are as follows:

If Program A is adopted, 200
people will be saved.

If Program B is adopted, there
is a one-third probability that 600 people will be saved and a
two-thirds probability that no people will be saved.

Which of the two programs
would you favor? [Consult audience]

In
this version of the problem, a substantial majority of respondents
favour programme A, indicating risk aversion. Other respondents,
selected at random,

receive
a question in which the same cover story is followed by a different
description of the options:

If Program A is adopted, 400
people will die.

If Program B is adopted, there
is a one-third probability that nobody will die and a two-thirds
probability that 600 people will die.

What do you say now? [Consult
audience]

A
clear majority of respondents now favour program B, the risk-seeking
option. There is no substantive difference between the versions to
justify the change in judgement. The phenomenon, which has passed
into common discourse, is called framing: how a question is
presented affects our judgement considerably. It affects experienced
professionals like doctors as well as the hoi polloi. The
effect is also why politicians don't answer the questions Jeremy
Paxman puts to them – it's not dishonesty, they are trying to
control the framing.

Experiment
5

We
have time for one more so I'll go for a marvellous experiment where
the researchers took out panel ads for a period in two student
newspapers on the same Michigan campus. The panels included, without
explanation, vaguely Turkish-sounding nonsense words: kardirga,
saricik, biwojni, nansoma, or iktitaf. The frequency of
the words varied. When students were later asked about their feelings
about the words, those that had been repeated often evoked more
positive feelings (TFS, Ch. 5). Other experiments have shown
that a word or image or number can supply a mental anchor and prime
us to be more likely to respond soon afterwards in a particular way,
which would not have been news to David Hume. Repetition makes
certain. If it's familiar, it's better. This explains the fortunes
spent on advertising; it is not direct persuasion, but priming
by repetition to create a favourable attitude.

V
: Two Systems

There
are a lot more data in Kahneman's book – one of his articles
with Tversky listed 20 different forms of cognitive bias. I have
given you a big enough sample for you to judge the flavour and
solidity of their work. The most important aspect to my mind is the
theory they built to hold it together. This is of two mental systems
of judgement:

an intuitive mode in
which judgments and decisions are made automatically and rapidly,
and a controlled mode, which is deliberate and slower.

Kahneman
misses a trick by not tying these to the ancient metaphors of the
Hare and the Tortoise, which I propose to do. The intuitive system 1
(the Hare) is fast, automatic, unconscious and effortless. It can't
be turned off.

Just
how fast and automatic can be illustrated by a simple mention of two
unrelated words:

bananas vomit

In
the fraction of a second it took your minds to process these words,
three things happened.

1. A negative reaction of
disgust to the second word. Your heart rate increased and the hair on
your arms rose a little. Your face may have twisted.

2. A feeling of surprise, of
an incongruity between the two ideas.

3. You started searching for a
story to resolve the incongruity, perhaps rotten bananas. This
happened in spite of my telling you the words were unrelated.

You
had no conscious control over any of this. For a short while, you
will look differently at bananas – don't worry, it will fade.

For
our purposes, the problem with the Hare system is that it cuts
corners. Most of the time its quick-and-dirty Brummagem methods serve
us well. But not always; we have to learn to steer into the
skid, or centering the rudder in an aircraft spin, these actions do
not come naturally. Kahneman is your guide to its quirks. It is worth
spending time to learn more about them, it will help your inner
Tortoise to stay on top.

Some
problems the Hare system can't handle at all. What is 17 times 24? No
even approximate solution comes to mind. It has to be worked out,
using a machine or a laborious algorithm we learned at school. The
emphasis is on work. The Tortoise System 2 takes effort, and
we don't like effort. The Tortoise also acts as a censor or pilot
instructor for the Hare, and is capable of taking over.

The
evolutionary story behind the double system is very plausible. Our
minds evolved for the life of bands of hunter-gatherers on the
African savannah. This is a complex and dangerous place. To stay
alive, negotiate your place in the band, and hunt game and
search for fruit and
edible roots successfully
you need to take a lot of snap decisions. Is that shadow a lion?
Which way will the warthog run? Is the dominant male's snarl a real
threat or for show? But these were not the only problems our
ancestors faced and solved. Some of them hunted hippos: a very large
and aggressive animal that spends most of its time in water.
Palaeolithic men and women and their dogs sailed to Australia. Later,
Polynesians learned how to get food from the seed pith of sago
cycads, which is extremely poisonous without repeated washing. You
can't do these things by intuition, only careful planning, and
in the sago case, experiment.

VI
: Limitations and conclusion

The
work I have described to you is important and solid. It does not
cover everything. We may not yet have the full list of intuitive
biases. More important to my mind is the fact that System 2, the
Tortoise, also errs. The errors of systematic reason were baptised
Idols of the Theatre by Bacon, and they are still waiting for
their modern Kahneman. There are many examples in the history of
science and medicine of learned men clinging to refuted theories; and
more in the history of politics and religion of cruelties and follies
carried out in the name of complex schemes of ideas owing everything
to the Tortoise system 2. At times, they have overwhelmed ordinary
moral intuition. We mustn't give food to the starving Irish peasants,
it will only encourage their fecklessness and unsustainably large
families, doctrinaire English liberals said in the 1840s.

Kahneman's
book actually includes one experiment that sheds light on such
aberrations. Researchers shot a video of a simplified basketball
game, which you can watch on the Internet3.
From the website:

You are asked to watch a short
video in which six people - three in white shirts and three in black
shirts - pass basketballs around. While you watch, you must keep a
silent count of the number of passes made by the people in white
shirts. At some point, a gorilla strolls into the middle of the
action, faces the camera and thumps its chest, and then leaves,
spending nine seconds on screen. … When we did this experiment
at Harvard University, we found that half
of the people who watched the video and counted the passes missed the
gorilla. It was as though the gorilla was invisible.

Counting
the passes calls for intense concentration on a difficult task, under
conscious control. It looks as if this focused effort sucks up all
our mental energy, or cashes all our attention budget, incapacitating
our normal background awareness. I suspect a similar process is at
work when the intense conscious effort of absorbing a complicated
ideology asphyxiates our normal moral empathy and intuitive
judgement. We become the superior intelligences that have passed
beyond the unthinking prejudices of the common herd. In fact, we have
sunk below them.

So
stay suspicious. Let the Hare and the Tortoise keep a good eye on
each other. Slow down in judgement when it’s possible. It’s
much easier to do this than to change our reflexes or emotions.

A
final and more cheering thought. Kahneman underlines the strain of
thinking, conscious Tortoise system thinking, which well explains why
so many people seem to give it up. But it does not explain the Nerja
Philosophy Group. You come here regularly and entirely of your own
free will for a couple of hours of demanding mental effort. Thinking
can be fun. It is like physical exercise: once you get into the
habit, you don't want to stop. The couch potatoes don't understand it
at all.