I have been involved in high tech, graphic arts, computer software and hardware design for more than 40 years.

I've been blogging about vaping since early 2009.

I work on advanced robot vision and software technology.

I own my own business.

I have set up this blog to talk about who I am, what I do, and to publish my opinions...

Search This Blog

Tuesday, January 3, 2012

The "Linda" Experiment

In "Thinking, Fast and Slow" by Nobel laureate Daniel Kahneman writes about what is known as "The Linda Experiment."

Various individuals and groups are shown the text below:

Linda is thirty-one years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.

Then each individual or group is then asked "Which alternative is more probable?"

Linda is a bank teller. (1)Linda is a bank teller and is active in the feminist movement. (2)

Respondents overwhelming choose (2) over (1) even though (1) must be more probable than (2).

(The reasoning being that if I have a group, say of 100 people in a room that are bank tellers. By definition subgroups of those people "bank tellers" and, say, people with red hair must be equal to or smaller than the group of 100, i.e., there are also blonds and brunettes in the group. You can also imagine this as a Venn diagram: a large blob of bank tellers and, within the larger blob, a smaller blob of those who are "bank tellers" and have red hair.)

I don't like this and while I understand the strict sense of what Kahneman is saying and doing mathematically I think that his results are not clearly resolved. I have been thinking about what this means.

Kahneman's point here is that the human mind does not process the notion of "probable" in a mathematical sense. In fact, this is one of the premises of the book.

But Kahneman's also speaks about the notion of associative memory in this book, particularly in relation to "System 1" which is the part of the mind that is the first to process information like this. Certainly probable has meaning in other contexts than the purely mathematical one Kahneman uses...

So what is an "associative memory?"

We can divide types of "memory" up many different ways depending on the discipline involved. From a computer-oriented notion (or mathematical logic sense which is what I use) we can say, for example, that "regular memory" is based on an index, such as a number:

1) Fred
2) Barney
3) Wilma
4) Betty

So when we ask what's in memory at 2, i.e., do a "look-up for 2" we get "Barney."

We can then describe an "associative memory" as something that works like this:

A "look-up" in this case supplies data to match in one or more columns, e.g., "Male" and "Blonde". We could also supply subsets of data, e.g., just Sex as "Male" in which case we would get two answers.

The basic notion is that we can "associate" a given attribute type (Male) with specific values to find something in our heads. "Oh, that looks like my mother's old ..."

We can also think of this in reverse, as in "What are Fred's attributes?" In this case we know Fred is Male and Fred has Black hair. So look-up in our minds can work in both directions.

In computer systems special hardware is often designed which makes this kind of look-up very, very fast. It is thought that the human mind is especially good at associations and associative look-ups functions.

So we can now think about the Linda problem using mathematical logic and sets in the same way as the associative memory system I described.

So this set of things represents in our minds an index in an associate memory for Linda. Linda might come up in a discussion of politically active friends, or in a list of people who went to college, or in reference to someone who is involved in activities against nuclear power.

The reason, of course, is that each of these things would be part of an associative look-up, e.g., "politically active friends" = { { Politically Active, Yes } }, and so on as we described with Betty and Barney above.

Logically the notion of "politically active friends" turns into a question that says is it true that "there exists an entry in memory such that the Politically Active attribute is Yes? (We're going to skip the mathematical logic/algorthmic version of this because entering the notation into a web page is too tedious.)

So now let's think about Kahneman's experiment.

Our associated memory is going to get loaded with Linda from the text Kahneman provides - presumably we don't know Linda.

Next we are going to read about the two choices. Looking at the stereotypical info we store for bank tellers its not likely we are going to have much cross over match with Linda - though we could if we were in some way familiar with them.

Now let's look at the "feminist movement" part of Kahneman's question.

Likely it will bring up the set of things that we think about for feminist movement (at least in the stereotypical sense). Note that there is very likely (as born out by the results Kahneman reports) a good number of associative elements, e.g., Politically Active, that will be brought to mind as a result.

Now here we see what happens with the conjunctive "and" in an associative memory world.

When we search our associative memory for Linda we will bring up all her attributes. The word "and" causes our minds to also bring up all the attributes we associate with "feminist movement."

In doing so we see "and" as describing additional attributes of Linda, i.e., Linda's attributes plus the fact she is "active in the feminist movement". Our associative memory converts "active in the feminist movement" into its list of attributes.

From an associative memory perspective there is then a much large set of matching attributes for Linda (assuming bank teller attributes don't match much with Linda to begin with). Those of Linda and those with "feminist movement." But our minds, I think, combine the two sets of attributes with an "intersection" finding what attributes are shared between Linda and activity in the feminist movement. This match to our minds makes the "probability" that Linda is active in the feminist movement because its attributes match in that context with Linda.

Our minds don't easily distinguish the difference between mathematical probability in the traditional textbook sense and the notion of probability in the sense that "the more attributes that match (are common) between Linda and activity in the feminist movement" the "higher the probability."

So what I am saying is that, from the perspective of associative memories and mathematical logic, there is ambiguity about what "probability" you are being asked to consider.

Our minds map attributes and match them quickly and by providing an additional set of attributes about Linda we are fooled into considering the increase in matching attributes as "probable."

While Kanheman says "probability" with the notion of strict mathematical and statistical probability I think that rather than drawing the conclusion that our minds are weak in that strict sense the results instead should be a good example of how in fact we associate between things.

I think that's what our minds do - associate what our senses tell us in order for us to draw useful conclusions.

Instead of the questions Kahneman asked we could instead ask what I see as a more revealing set of questions:

Most would answer, I think, that there would be a good chance that (1) would. And this question would use the exact same mental machinery as Kahneman did. For choice (2) basically the question becomes a non sequitur because we no nothing from the narrative about what Linda does for a living.

Probability in the mind of the untrained statistically means the probability of an associative match I think.

I think that Kahneman is fooled by his own concept (or bias) of what "probable" means.