DANIEL KAHNEMAN: I want to tell you a bit of straight psychology that I find very exciting, that I found more exciting this year than I had before, and that in some ways is changing my view about a lot of things in psychology.

There are two big things happening in psychology today. One, of course, is everything that's got to do with the brain, and that's dominating psychology. But there is something else that is happening, which started out from a methodological innovation as a way to study memory, and we've always known, that's the idea of the notion of association of ideas, which has been around for 350 years at least.

We know about how associations work because we have one thought, and when it leads to another‚windows and doors and things like that, or white and black‚and we have our ideas of associations, and it's always been recognized as important and interesting. But our view of how associations work has been changed in a profound way by a technical innovation, which is something that happens a great deal in psychology and I suppose in all sciences.

This innovation is the following: If, for example, you hear the word "sick", there are few associations that come to mind. But there are a number of other things that you can do, that are little more refined. You can present words, and measure the amount of time that it takes people to read the words. Or you can measure words and non-words, and the task is to decide whether they're a set of letters, or a word, or a non-word, and it's the ease with which words are recognized as words as against non-words. I'll begin by focusing on reaction time, because that's the simplest one.

Here's how it works: after the presentation of the word "sick", the number of words that are affected to which you react differently than you reacted before is enormous. You will be faster, obviously, to "ill", and to "death", and so on, but it will be "hospital", and it will be "nurse", and it will be "doctor", and all of a sudden you've got a huge range of things to which the response is influenced by just presenting that one thing. We find that associative networks that we have in our minds appear to be a lot richer than it did before.

But that's not the only thing that happens. It turns out that the kinds of associations that are built in are a lot richer than we thought. When I want to make an impression while giving a talk, I put the word "vomit" on the screen. Just imagine the word "vomit" on the screen. I point out we know from experiments what happens to people within the first second or two that the word "vomit" is present. In the first place there is that enormous range of changes in the associative structure, and the readiness as well.

But there is more that happens. You have a facial reaction. When people see the word "vomit" or hear the word "vomit", and you can take pictures of their face, you will see that the reaction is that they're disgusted. It's more than that. Nobody noticed that this was happening to you, but you recoiled; you recoil when you are presented the word like "vomit".

I'll give you an example of the kind of experiments that demonstrate it. I find them very elegant. You put people in front of the computer screen, you give them a lever. You tell them that things are going to appear on the screen. It doesn't matter what appears on the screen. Mostly there will be words, but it doesn't matter. What you're supposed to do is as soon as anything happens on the screen is to move the lever and make it go off. It's completely independent of content. You present words. The experimental manipulation is that for half of the subjects the lever moves this way; and for half of the subjects, it moves that way, towards me or away from me.

You present words that are either good or bad‚"peace" and "love" versus "crime" and "death" and "sickness" and "vomit", and what have you. It turns out that the speed with which people respond to the lever, to the words, although the word is irrelevant to the content and they don't have time to become conscious of it, the speed with which they respond depends on the lever. They are faster, much faster‚all of this is measured in tenths of milliseconds, the effects‚ but they're robustly faster pulling the lever towards themselves when the word is good, and pushing the lever away from themselves when the word is bad.

It's not the real thing, it is a symbolic representation of the real thing, and people are responding to the symbolic representation, in a way as if it were the real thing. It goes on from there. I'll now give you a few more examples. That's called a priming paradigm. But the word "priming" in this context is like priming a pump, and you'll see it the most clearly in the case of recognizing a word that is presented. You are primed, you are ready, to recognize that word; more ready than you are to recognize other words.

The thing that happens is you can present a smiley face so fast that people absolutely do not see it, and it influences their responses to other things. They tend to like everything better if the smiley face has been presented, which they're not aware of. Consciousness is not necessary for the effects. What began this whole thing‚and it's one of the most important contributions of psychology in the last decade or so‚is a study in 1996 associated mainly with the name of John Bargh who is at Yale.

Since then, there has been an explosion of research. Bargh was at NY at the time, and in the study that kicked off this line of research, people were brought to participate in an experiment and were given one of the tasks that psychologists use to make you think of words. You might have a set of five words, or you have to take four of them and make a sentence out of them, or to memorize lists of words. Different groups of subjects are exposed to different lists of words that will be used as primers. And one of the lists contains words like "wrinkles", "Florida".

The word "old" is not mentioned. Then they're dismissed from that experiment, the experiment is over, and they're sent over to participate in another experiment, which is at the other end of the corridor. The dependent variable is the speed at which they walk, and you have a substantial effect on the speed. You haven't mentioned the word "old", and you made those people act as if they were old. You've primed "old", and you've primed the behavior of an old person.

It works the other way, too. That's an experiment done in Germany. You take people and with a metronome you make them walk, and you make them walk slowly. The metronome just beats slowly, so they walk slowly. They're faster to recognize any word that has to do with old age. You haven't gotten anything, they're just walking more slowly than usual.

What you get is the point I'm trying to make; we speak a lot about logical coherence, that's what the rational agent was supposed to be, the rational agent is supposed to be coherent. There is remarkable coherence in those results. What you end up with as a result of the presentation of a single word like "vomit", you get a lot of changes that reinforce each other. You get an emotional response, and you get a facial response, but the facial response primes emotions.

I'll give you an example of the kinds of experiments that people are doing now. You give people a pencil, and have them watch cartoons. First, they watch with a pencil in their mouths horizontally, and then with a pencil in their mouths sticking straight out. And they are rating how funny the cartoons are. Cartoons are a lot funnier if you have a pencil in your mouth horizontally, than if you have the pencil sticking straight out. Nothing has been mentioned about mood or anything else. You are creating a facial shape that is the shape of a smile, or more of a frowning shape. That influences emotions.

A famous experiment being done is, you give people an earphone. This is supposed to be an experiment on the evaluation of the technical quality of earphones‚ and you say, "We want to know how those earphones respond to different movements", and you have people listen to messages, persuasive messages of various kinds, while either they're told either to nod their heads, or they're told to shake their heads. Big effect. You nod, the message looks more believable. You shake your head, the message is less believable.

What you get is, it seems as if everything is interconnected, and is interconnected in a very coherent way so that you should see what the biological significance of this must be. The biological significance is that you are prepared, and you're prepared in a funny sort of way. Why would the word "vomit" make you prepared for emergencies? But the word "vomit" triggers a readiness to flee or fight. It's a threat, and you're responding to it as a threat, and you're responding pretty specifically, which is you are prepared for certain kinds of bad things that are related to that threat more than you are prepared to react to threats in general. It's specific, and it's internally coherent.

You get those strangers when it could go any differently, because there have been hundreds of those experiments, and some of those are pretty huge. You mention the library, and you measure that‚which you should have done last night, if you were complaining about your too loud neighbors‚you mentioned the library, and it softens people's voices.

What you get from this is a sense of where the control of behavior is, and the control of behavior isn't where we think it is. All behavior turns out to be a lot more controlled by the environment.

MYHRVOLD: Okay, so you have, both last year and now, you have these very interesting little examples. And it's amazing that, in fact, the effects are strong enough to be measured. But ultimately, how strong are they? Because if, in fact, there is this insane context dependence, where we're sort of willy-nilly making wildly different decisions every point by point, that seems unlikely.

KAHNEMAN: No. Not wildly.

MYHRVOLD: Is this a background thing that's very small in the grand scheme of things?

KAHNEMAN: What happens is you can take any phenomenon and amplify it by focusing on it, and this is what we do in experiments.

MYHRVOLD: Sure.

KAHNEMAN: But the general idea would be that in the real world there are lots of those slight, subtle influences that are working. They are very diverse. Some of them are not so diverse. I'll get to some examples where probably the influence is pretty general because the culture is systematically priming you in some directions or in others. If you're in a culture that provides environments that are specifically primed for certain things more than other things, it is going to make a difference, and apparently there are examples where it does make a difference.

MUSK: Have you done studies on the effects of say subliminal versus explicit advertising? Where you show some subtle image of a Coke, or whatever it is, and then you do an explicit ad for a Coke, and you see how many people go buy Coke?

KAHNEMAN: There was a classic claim that was made very, very early on, and it was false. The early claim was false, and then the belief was, oh, there is nothing to it, and then in the last decade it turns out may be that claim was false, but you can get more dramatic effects quite readily, including the effects on behavior, and I'll give you examples.

MUSK: I'm just wondering about the difference between the subliminal suggestions versus an explicit.

KAHNEMAN: There are many situations in which subliminal effects are stronger than superliminal effects.

MUSK: Like what?

KAHNEMAN: Well, in some of those priming experiments, if you see that you're being primed, you can resist it. If you don't see that you're being primed, there is no way you can resist it. You sound, France, as if you know of examples‚ I'm blocking now, but believe me, there are examples like that.

LECLERC: Yes. I certainly agree with you that the first study was claiming that the first study that came out was with Coke.

KAHNEMAN: Yes, that was the hidden persuader. That was false.

ROMER: But if you take the smiley faces, does the smiley face have a bigger effect if it's subliminal.

KAHNEMAN: I think so. It certainly is not weaker, and possibly it is stronger. Reminding people of death, just of the fact that they're mortal, changes their responses to many things. Among other things, they're more responsive to the American flag, and they tend to be more Republican in their attitudes. But that's just reminding people of death, reminding people of mortality

MYHRVOLD: You can make Democrats and Republicans by threatening them with death? That's fascinating!

KAHNEMAN: Those effects would be small at the margin, but there are those effects that are small at the margin that can change election results.

You call and ask people ahead of time, "Will you vote?". That's all. "Do you intend to vote?". That increases voting participation substantially, and you can measure it. It's a completely trivial manipulation, but saying 'Yes' to a stranger, "I will vote" ...

MYHRVOLD: But to Elon's point, suppose you had the choice of calling up and saying, "Are you going to vote?", so you prime them to vote, versus exhorting them to vote.

KAHNEMAN: The prime could very well work better than the exhortation because exhortation is going to induce resistance, whereas the prime ‚the mild embarrassment causes you to make what feels like a commitment, and the commitment, if it's sufficiently precise, is going to have an effect on behavior.

THALER: If you ask them when they're going to vote, and how they're going to get there, that increases voting.

KAHNEMAN: And where.

Here is a study, this one will demonstrate that those effects are not so weak. You look for support for school bonds, and what you look for is where were the polling stations. When the polling station is in a school, you get measurable effects on the support for school bonds. They increase. That is non-trivial, it's in the real world, except that you have something that is focused, you know what the direction is. You expose a lot of people to the prime, and you observe the behavior, and it's quite measurable.

THALER: And to answer your question, on that one, my recollection is the magnitude is something like 2 or 3 percent. It's not a huge effect, but a noticeable effect.

KAHNEMAN: Yes, that's what you would expect.

MULLAINATHAN: There is another response to this question. And I've struggled a lot with this question. If these effects are so big, how can it be, right?

There is another more controversial response to that, which is, let's say that the two phenomena that are opposing each other is that people are relatively consistent and stable, but these effects suggest a lot of instability. One resolution to that is that, in fact, people are not consistent and stable ...... and, the bias is that we think ourselves and others are consistent and stable when we're not. There is good evidence that if you take even something as simple as stated preference for Democrat, Republican, test-retest validity on these things is tiny, risk aversion measures have tiny test-retest validity. One possible resolution control of this is that the mistake is on our end in presuming stable interpersonal characteristics.

KAHNEMAN: That's a beautiful way of putting it, because one of the things that psychologists have been exercising over and over for decades is the relative impact of personality, if you will, or character or temperament—internal factors as against environmental factors in the control of behavior. We have a hugely powerful bias against the environment as a determinant of behavior. We tend to believe that somebody is behaving that way because he wants to behave that way, because he tends to behave that way, because that's his nature. It turns out that the environmental effects on behavior are a lot stronger than most people expect.

MYHRVOLD: Do you know Peter and Rosemary Grant at Princeton? They're just retiring, or just retired a couple of years ago in the biology department, and they studied Darwin's finches.

KAHNEMAN: Oh, yes. I know about that.

MYHRVOLD: The received wisdom prior to their study was that evolution is very slow. And they showed that that's completely false; evolution is incredibly fast and dramatic, it's just that it's driven stochastically. And so, in fact, the net movement, if you sample infrequently, it looks slow because you have this very unstable thing. But if you track it year by year, as they did, it's very fast. That would be an example of saying in fact it could look very unstable, but with sampling it at the appropriate frequency against a stochastic background, it doesn't.

KAHNEMAN: The question is what's appropriate? If you want to understand the phenomenon, you sample high frequencies, not low frequencies.

MYHRVOLD: Well, they're the first people to ever study evolution at high frequency, because everybody thought it wouldn't work.

KAHNEMAN: It moves year by year because it's the shape of the beak that determines which bird survives. And so there is a large selection effect, and it's immediate. It's happens year to year, and it varies year to year because it depends on the weather that year.

Another condition is a pile of Monopoly money on a neighboring table. What would you guess, by the way? Those of you who have read it shouldn't guess, but can you guess? I was stunned by the result, which I wouldn't have predicted. But can you guess what priming people with money will do?

They don't want help. They're on their own. They also don't want to give help. You've got very clever ways of manipulating that, of observing that, but my favorite is the experimenter that comes in clutching a batch of pencils, and the pencils drop on the floor. The dependent variable of the study, the number of pencils the person picks up, is fewer if there is money on that screen saver.

You prime Woman. You get a significant deterioration of performance. You prime Chinese, they do better on mathematical tests. That effect, by the way, is small.

What is happening I call the poetry of priming.

MULLAINATHAN: It acts on what?

MYHRVOLD: Makes them Democrats?

HILLIS: Republican.

KAHNEMAN: It's closer to making them Republican. It makes them individualists. And it's quite deep, and very unexpected. It doesn't make them good or bad, it just makes them different.

That's a favorite manipulation among psychologists, you give people an anagram that they cannot solve. They probably can't solve it. But they are told that if they run into trouble, they can go and ask someone for help. And you measure the time that it takes people to go and ask for help, and just the screen saver in the background almost doubles it.

They don't want help. They're on their own. They also don't want to give help. You've got very clever ways of manipulating that, of observing that, but my favorite is the experiment that comes up with clutching a batch of pencils, and the pencils drop on the floor. The dependent variable of the study, the number of pencils the person picks up, is fewer if there is money on that screen saver.

MYHRVOLD: Time is money.

KAHNEMAN: Significantly fewer.

THALER: That's why economists are jerks.

KAHNEMAN: You tell people that they are going to be participating in a conversation exchange with somebody else, and they're supposed to set up the two chairs for themselves and for the other person. The dependent variable, the distance between the two chairs, increases.

MYHRVOLD: It's your surmise that all of those things can be lumped together under individualism?

KAHNEMAN: No. You're looking for what all those things have in common. I don't know where she came from with her hypothesis, but clearly that's what these things seem to have in common. They increase the distance between you and other people. And they make you less willing to get help, and less willing to give help. For those who are interested in cultures, that is like a cultural effect.

HILLIS: When you tell us about these experiments, have they been replicated?

KAHNEMAN: Yes.

HILLIS: They're very robust?

KAHNEMAN: The first response to any of these experiments is to say, "I don't believe it". And that was certainly our response initially, and there are experiments where the results are too extreme. I might tell you a couple that I just can't believe ... but they're so consistent, and it's not one person, and it's not one lab. There are dozens of labs, and dozens of people, and hundreds of post docs. The rate of success in those experiments is at least comparable to the rate of success in other domains of psychology.

I belong to a conversation group associated with the Center for Rationality, Jerusalem, and somebody from Jerusalem reported on what are the results of showing people a flag. It turns out that has a big effect on people's behavior, completely unexpected, by the way. I would have thought it pushed them to the right. But it pushes them substantially to the center. The claim was it affected voting behavior, which I found hard to believe, but somebody said, "I don't believe it", and we had an interesting conversation, because I said, "You don't have the option not to believe". Believing is not optional. If you accept that this is science, and this is replicated science, then belief is obligatory. You may doubt one result or the other, but it gets futile to doubt a particular result because there are so many of them.

MYHRVOLD: No, you could have the following, you could say, If, in fact, we are so sensitive to priming, then you have to be extraordinarily careful with your design of the experiment, because the same experiment that had the screen saver in the background ... what if the experimenter wore a red shirt, and, oh, my God, red shirts! You know what those do, right? You get this funny thing that the more you claim were insanely sensitive to boundary conditions, the more you have to be careful of experimental design, don't you?

KAHNEMAN: That is not a big deal. Let me explain why. Because you would assume there is one manipulation that you've picked. You're going to use the word "old", or not use the word "old", and then the shirts, they vary at random, or they're the same for everybody. There is a lot of background noise that should increase the variance, and it should make it more difficult because they're not controlling the thing. It should make it more difficult to get those effects, provided that the rest is randomized.

MYHRVOLD: Provided the rest if randomized. But, in fact, there could be a systematic thing.

KAHNEMAN: But it would have to be confounded with the experimental manipulation. To produce the experimental result, you would ...

MYHRVOLD: But that's the question about replication.

KAHNEMAN: And that's what I'm telling you, don't worry about.

MYHRVOLD: Okay.

KAHNEMAN: This has been done too many times.

THALER: The experiments have been replicated, or there have been lots of ... because the other worry they have is, what is the publication bias?

KAHNEMAN: The old age thing has demonstration quality. You get priming effects that are socially significant. Take Chinese women mathematicians at Harvard, and the dependent variable is going to be that they're going to be given a test of mathematical problems, and the procedure is, the manipulation is, what questions are they asked to identify themselves prior to that. And you get one group who were asked basically about their ethnicity, so you prime Chinese. And you get another group that are primed for gender. You prime Woman. You get a significant result. You prime Chinese, they do better on mathematical tests. That effect, by the way, is small.

HILLIS: I trusted you in the experiments until Nathan flashed that dollar bill.

KAHNEMAN: That had no affect at all. Okay, interesting.

TREISMAN: The experimenter wore the same shirt in both groups.

MULLAINATHAN: An interesting place where you can also feel this in yourself, and it's hard to see these things, is in these implicit attitude tests. These are things where you use the nature of priming to measure implicit attitudes you may have. And these tests are amazing, you can do them on the Web, and when you try them, you can feel the power of certain primes physically. You can just feel it's just that little bit slower to respond ...

KAHNEMAN: It's very straightforward.

THALER: In the shooter study, which you can do at the University of Chicago, the task is you're shown an image of a person on the screen, and they either have a gun or a cell phone, and they're either white or black, and your job is to shoot or not. And you have to shoot fast. And you shoot more black guys with cell phones, and white guys with guns kill you more often.

MULLAINATHAN: And you feel it. That's what I find interesting. Go to Implicit.Harvard.edu, and you can try these. And what I find amazing is you can do this in class, and you don't even need to go through the whole experiment. Students just stop because they feel that it's just a little harder.

KAHNEMAN: The experiment to make it more vivid is, you get words that could be either good or bad words, and they come with faces that are completely irrelevant. The task is to respond to the words, but the words compared with faces, and many people are slow to respond to good words if they're paired with a black face than if they pair with a white face, and those are people who consciously are, they abhor discrimination. But you can show that. That's what Sendhil is talking about.

Let me tell you about another experiment of that kind, and here I'll show you the size of the effect, which I find astonishing. You can see for yourself. This is a laboratory‚ I forget in what field‚ in the U.K. where, as in many labs in the U.K., they have an honesty box, so people can go to get their coffee, or their tea, and there is an honesty box, and there are suggested prices, but nobody supervises. In the experiment they put a poster above the coffee, and they had two sets of posters, and they alternated them every week. One was of eyes, and the other was of flowers. These are the results. This is dollars paid per liter of milk consumed. And the poster showed next to it. This is not a small effect.

THALER: No, but those were real eyes.

KAHNEMAN: That is good.

BEZOS: The male eyes are the biggest, too. Is that based on statistics, or is this just a little sample?

KAHNEMAN: That I have no idea.

MYHRVOLD: No, but it's by a factor of two. The alluring female eyes aren't much better than flowers.

THALER: Well, that guy at the bottom is pretty scary.

KAHNEMAN: Yes.

THALER: I would pay double ...

MULLAINATHAN: You know who that is? That's Golem from "The Lord of the Rings".

KAHNEMAN: I'll tell you now about an experiment that I find fairly remarkable.

There have been differences established between the way that Asians, especially Chinese, perceive things, as against Americans, or Westerners in general. There are systematic differences that show up in experiments. There was a nice book that appeared a few years ago called The Geography of Thought by Richard Nisbett, one of the great social psychologists of his generation. He points out that the main difference is that the Chinese see things much more globally than the Americans do.

That has to do, again, with community versus individual. When you show them a complete picture, they are more sensitive to things that are in the periphery and they're less likely to pick out individual figures, and more likely to pick out groups.

A lot of speculation has gone into this. Until somebody, to the great irritation of Dick Nisbett, showed that you could turn people into Chinese in about six minutes, and the way that she does it is she has people read a story aloud. And the story's about spending an evening in the city. It started, "We went to the city, we did this, we did that, we'd open a window..." and it ends, "the city was ours". The other is identically the same words, except it's 'I', and it ends with "the city was mine". The test uses a standard psychological measure, you show people a letter made up of letters, and you ask them either to respond to the big letter, or to respond to the interior letter ... and after saying "ours", the community manipulation, people are more like the Chinese, they find the large letter relatively easier.

MYHRVOLD: Forest for the trees.

KAHNEMAN: And vice versa, after being primed.

I'd like to wrap up and give you an example that I find striking. It's one of the last I heard about, in a way I find it more bewildering than most of the others. Although I'm pretty confident telling you about those, I find this bewildering.

In this experiment, you give people a grid, and you give them coordinates of two points, and they draw coordinates of two points. In one condition these are the two points, and in the other condition these are the two points. That's it. And then you test them in other situations. The whole experiment is drawing two points.

What is happening I call the poetry of prime, because this is telling us something about the way the mind works. Marking off the two distant points by their coordinates, they tell people a shocking story about an obese woman, very fat. She reads a book, and in the book there is something written by her former lover who speaks about what it's like to make love to a fat woman. It's not a nice story. The dependent variable is how intolerable you find it. Marking off the two distant points, they find the story funny, where as if the two dots are close, they find it almost intolerable.

THALER: This one I don't believe.

LECLERC: I don't, either.

KAHNEMAN: There were a series of experiments along those lines, telling people a disgusting story, and their ability to tolerate the disgusting story increases. What that tells you, and they have a lot of evidence of this in the case, is that the way our mind is built, we have metaphorical connections that are very powerful. We perceive metaphors. We have a very limited ability to control our minds, and that's where I want to go with this.

What this is telling you is that when we see distance, which was a physical distance, we understand that at multiple levels, including symbolically, and including emotionally,... I was talking earlier of coherence, this is pushing it a little further. It turns out what you are learning from these experiments is an explanation of the major result that Amos Tversky and I obtained a long time ago when we studied judgment.

Our conclusion when we studied judgment was that when you ask people a question like what is the probability that something happens, they tend not to answer that question because they don't know how to do probability. They know how to judge similarity. To give you an example, if you tell people "Julie is now a graduating senior at Yale, and she read fluently when she was 4, what's her GPA?" People have no difficulty whatsoever making a guess about her GPA. They pick the GPA that is as impressive to them as reading at age 4. They just match.

That turns out to be wrong statistically, you're not supposed to do that. They don't know that they're doing something that they're not supposed to be doing. You ask people what is the probability that Julie will be in the top third of her class. They will answer thinking they're judging probability, when they're judging something else, and they don't know they're doing it. That seems to be a very general way for people to make judgments and to make decisions. That is, you ask people to do one thing, and they do more than you ask them to.

I'll give you another example of that; , maybe two. This result is an old one, but now you've seen how it fits with the priming. You tell people, I'm going to give you sentences, and as quickly as possible, you will press one key if the sentence is literally true; and press another key if the sentence is literally false, or is not literally true. And then you present sentences like "Some jobs are jails", "Some roads are snakes". People are slow to say these are false. They have computed the metaphor. You didn't ask them to compute the metaphor. When you ask people to decide whether something is true, they're doing all sorts of things in their heads, including evaluating the metaphor. It doesn't make sense. Now, it's more than that.

You can manipulate whether a statement appears true or false to some extent, not completely of course, by the font in which you print it. You print something in a font that is highly legible, it looks true. You print something in lousy print, it looks false. You make something familiar, like you present a word, you see to it that people repeat or are exposed repeatedly to the expression "chicken heart", let's say, and then there is a statement that says "The chicken heart beats at 343 beats per minute, true or false". If they've seen "chicken heart" a lot, they'll say "yes" to almost any number that you suggest. The whole thing looks familiar.

Now, I know you don't want to believe me.

MYHRVOLD: I'm trying not to.

KAHNEMAN: There are effects of the pronounceability of stock ticker names‚ the three letters that make up the symbols in the stock market. It's an effect that doesn't last, but in the short run it's highly significant. A pronounceable ticker name is good for the sale compared to a non-pronounceable name.

The font in which you present stuff is also important. You present names of cities to Princeton students, and you manipulate the font so they see some cities in a lovely clear font, and the others are kind of blurry, low contrast. The question is how far away are the cities? They're farther away if the font is bad. Again, it's poetry. People don't know. They get an impression, and the impression is an impression over which they have very little control.

This is telling us something; we haven't figured out everything that it's telling us, but it's telling us about the control of the environment over behavior, which I now think is more extreme than what I used to think until even quite recently. I've always believed the environment controls a lot. This is very extreme.

MYHRVOLD: Do these effects interact?

KAHNEMAN: No, they've been studied one at a time.

MYHRVOLD: The next interesting question is, two at a time. How many can you have concurrently? If you have the fonts like that, and the dollar bill screen saver is in the room ...

KAMANGAR: Is there a real example of the name association, Dennis, like 'dentist'?

KAHNEMAN: Oh, yes: Dennis and dentists. Yes. Did you know that people named Dennis are more likely to be dentists and people named George are more likely to live in Georgia? This is a huge data set. The effects are tiny, but they're significant.

THALER: That one you don't want to believe, but it's is true.

KAHNEMAN: The effects are tiny, but they're highly significant.

MYHRVOLD: Say the man named "Dick".

KAHNEMAN: People tend to mate with other people who share the same initials. There are dozens of things like that. But those effects are tiny. They're just verifiable when you have a huge set. But the point I'd like to leave you with ... I don't know where this is going, but this is a different view of how humans operate, which is in a way a lot closer to Freud than we have been for 100 years, certainly for the last 50. And in terms of the balance of how much about what's going on in our mind we know, it is making that window smaller. We know less about what's going on in our minds.

BEZOS: Are there known ways of treating your brain more like a voluntary muscle, having more control over your brain?

KAHNEMAN: I don't know of any.

MYHRVOLD: If you understood how they interacted, suppose that there is only one that can be active at any one point in time, then in fact you could deprime yourself, deliberately.

KAHNEMAN: No, no. You can prime for several words at once, so we know that priming is not that you are holding to one thing in your head. You present a list of words, and they're all primed.

ROMER: But you'd love to know would these priming effects be stronger in a population of people who had done like 20 years of theoretical physics or 20 years of meditation, for example? Or does it not vary at all? Does how we use our brain influence how susceptible we are to ...

THALER: Well, there is Shane's test.

KAHNEMAN: But that could just be plain intelligence.

TREISMAN: There is exercise of self-control; it does increase your muscle. It wears it down in the short-term, but it makes you better at resisting temptation in the long-term. That's been shown.

BEZOS: What does?

TREISMAN: Exercising self control, and it generalizes across a lot of different tasks, like squeezing something, persisting with anagrams that are unsolvable.

BEZOS: And it does what for you?

KAHNEMAN: It makes you more likely to treat yourself to a cookie.

TREISMAN: No: in the short-term.

In the short-term you give into temptation more readily. In the long-term, you may get better at resisting temptation.

PARKER: What about doing years of research on priming?

KAHNEMAN: People who do years of research on priming are just as susceptible.

PARKER: Are you sure about that?

KAHNEMAN: I told the group last year that I've been doing stuff on judgment for 40 years and, Dick is my witness, my judgment has not improved. It's not different.

PARKER: That's not scientific, that's completely anecdotal.

KAHNEMAN: But experiment attempts to educate people.

MYHRVOLD: Your career is doing pretty well.

KAHNEMAN: Well, for experiments to educate, it depends on feedback, but feedback on what? If you don't know what you're doing, it's very hard to change it. If you're not conscious of what you're doing.

MULLAINATHAN: One of another places where this is quite counter-intuitive, I don't know if this is data, but when I was going to Mahzarin Banaji's lab‚they do a lot of work on implicit attitudes of race‚one of the things that seemed to come out is, because when you work on discrimination, there is so much talking about attitudes about "blacks being bad", et cetera, et cetera, more implicit attitudes towards race become much more polarized towards black and bad. Because I've done a lot of work on discrimination, I have a much stronger "black-bad" association than probably most people in the room, which is odd.

KAHNEMAN: I'll end now with one more comment because you've mentioned Banaji. An interesting application of that is the New Yorker cover cartoon on Obama. Everybody's first impulse on that is, "seriously, come on; it's obviously satire, and what could that do". If you believe in what I just said about feedback, this is bad for Obama because you don't control it: Obama-Muslim; his wife-terrorist; Osama bin Laden-Obama. It has created, it has primed you, it has changed your associative structure in ways that you have no control over, and none of which are good.

MYHRVOLD: Sure. But couldn't you argue that it would be even worse if this were the day before the election?

KAHNEMAN: Oh, yes. Oh, certainly I would.

MYHRVOLD: In fact, doing it now could almost‚

KAHNEMAN: No, it won't inoculate you.

MYHRVOLD: Isn't there a vaccination effect?

KAHNEMAN: It won't inoculate you because there is no feedback.

PARKER: But if you look at these effects ...

HILLIS: But how long do these priming effects last?

KAHNEMAN: It varies. Most of these experiments, the tests are done quickly. The tests on the effects of flags on political attitudes that involved an unconscious presentation of the flag‚ and that's a pretty extreme result.

MYHRVOLD: But everyone has seen a flag at some point in his or her life. If it had infinite time duration, too bad, okay? We're already done, and it's not going to get any worse.

TREISMAN: It saturates.

MYHRVOLD: Yes, exactly. This can't possibly be infinite.

KAHNEMAN: Of course not. They clearly vanish. Something that happens when you age, if you write, and this I can tell you from personal experience, you use the word "therefore" somewhere in the paragraph, or "however", and then you go on writing, and then you read the paragraph, the word "however" appears three additional times because you primed the word "however". It comes to mind more easily, and there it is. If you get really old, and I'm approaching that phase, when you reread it, you don't recognize the number of times you just repeated it. Then you'd better quit. But the effect is there.

MYHRVOLD: I don't believe this one.

KAHNEMAN: The effect is there even for younger people, except you inhibit it because you remember that you said "however". You lose the memory, you just get the priming effect. Another thing, and that's a direct application of this, if you give people health warnings, especially if they're old, you tell them "Don't do something", in their mind, it's "Do something". They lose the negative.

MYHRVOLD: Really?

KAHNEMAN: Yes, yes. There are certain warnings that you'd better not give people, especially old people, because the stuff is in there. And a lot of statements telling people, "I'm going to tell you something that isn't true", okay, then you tell them something that isn't true, and you wait a week; they're more inclined to believe it than if you hadn't told it to them in the first place.

PARKER: Is this the part of the talk where you go back systematically through all the things you told us, and said, "Well, actually that one was false"?

MYHRVOLD: Next week you get an e-mail: "Bullshit".

THALER: Well, we're going to wrap up. There's obviously a theme here to be continued tomorrow. If you want a say in what we do in the first session tomorrow, cast your vote.

MYHRVOLD: I have one question. The bounded rationality thing, which we made fun of early on, now you've gone through this thing of saying that, in fact, we're enormously susceptible to an unending series of priming metaphors, which affect us, which unless they're systemic, are a large amount of noise, and so the only thing that's left might be the bounded rational part. Or not? It's only systemic effect. If, in fact, there are tiny, tiny fluctuations, change things radically, then over a very large number of people, don't you average all that out?

KAHNEMAN: Sendhil, you should be talking. I'm voting strongly for describing your research, your South African research, because it turns out in real applications, for people taking loans for real money, things that shouldn't matter, matter a lot. Of course, everybody here is sophisticated and knows this. But even the sophisticated may be surprised at how powerful those effects are.

On "The Two Big Things Happening In Psychology Today:
A Talk By Daniel Kahneman"

Dick Thaler assumed that we already understood and accepted the standard models economic discussion making. When you think about it, it is very flattering—he assumed that we had internalized the standard models to such a degree that we needed to be talked out of them.

Quick reply to Nathan on priming—I believe that the comparison of priming with the curiosities of visual illusions sells it short.

A) The priming studies provide a mechanism that explains the major "heuristics and biases" effects.

1) Anchoring effects are (mostly) caused by the fact that when I ask you if the tallest redwood is more than 800 feet tall I have primed you to think of very tall trees, and the sample of trees you recover from memory is biased upward—compare to what would have happened if I had asked if the tallest redwood is more than 80 feet tall. What you think about redwoods is affected by mention of the number. (true even if the anchor is known to be random).

2) Other heuristics come to be used because activating a cognitive goal actually activates a whole cluster of goals. You ask whether something is true, and that activates judgments of whether it makes sense in context, whether it is metaphorically true, etc. You ask whether someone will be a good President and an enormous amount or irrelevant garbage questions get answered automatically. This is how you can get judgments of similarity being used as proxies for judgments of probability.

B) Priming effects tell us generally that we don't have much access to what goes on in our minds. They make some common ideas sound silly—like "voters who are concerned about the economy prefer Obama". The kind of associative coherence in which everything primes everything else within a bundle of reactions is a better model than this. Associative and intuitive coherence is different from logical coherence.

C) Priming effects may provide one of the mechanisms by which culture works. Some cultures provide the equivalent of constant reminders of money. Other cultures remind you that there are eyes looking at you. Some make you think in terms of 'we', others in terms of "I"—and those are just examples that I mentioned in that one random hour.

One reply to Danny Kahneman's reply about optical illusions and priming... After reading your response, I still wonder if the optical illusion analogy isn't quite apt. I say "wonder" because I don't really know and it would require more thinking—and probably some experiments—to really tell. The mechanism of priming that Danny gives in his reply is that the brain unconsciously retrieves (a computer scientist would say "pre-fetches") a context based on environmental cues. This context then indelibly colors the next thought. Even if you are told that the data (such as tree height) is known to be random. This seems entirely plausible, and I accept it (but I have to ask, is there direct evidence of this, or is it a surmise?). But isn't this very closely analogous to what happens in optical illusions? In that case the brain also automatically processes the scene in light of visual context and it reaches a conclusion. It is entirely subconscious and works even if you are told that you are looking at an optical illusion. This seems to me to be almost exactly analogous to priming.

Here is a classic example:

I can tell you the punch line up front, but it won't change the results. The lines are all the same length—but no matter how much you know that, you still see them as different lengths. The arrowheads on the end of the horizontal lines subconsciously change our estimate of the length of the lines. They are like giving a number when talking about tall redwoods.

There are many optical illusion sites—but for quick reference here is a very nice one that has LOTS of quite different effects—http://www.michaelbach.de/ot/

Of course any analogy is only that—an analogy—and it will break down somewhere. Priming is not identical to optical illusions, but both seem to be due to unconscious processing by the brain, which is highly context dependent. I've read that there are multiple kinds of priming—conceptual and perceptual—and this may not apply to all of them (but I bet it does). I think what Danny doesn't like about this analogy is not how closely it applies, but rather the inference that priming might only be as important as optical illusions—i.e. as "just" an optical illusion. That is a fair comment, since that was partly what I intended. But not as a condemnation of priming—what I wrote is that "I wonder" how close the analogy is. Now I could be foolish for confessing this, but I will admit it: I wonder how important priming is to everyday life, or, in keeping with our theme, to economic life? Rather than wonder indirectly I'll put this as a question to Danny—how important is priming to everyday life, or economic life? How could that be measured?

The processing that goes on in optical illusions exists because it often gives the right answer—often enough that it is very useful. In particular many of the seeming paradoxes in optical illusions arise because the visual system is trying to solve what we know mathematically to be an ill-posed problem that has no unique solution. One example is inverting a 2D picture into a 3D shape. The simple classic is a Necker cube.

This is a perfectly good 2D picture, but we cannot help trying to force into being a 3D object. The 3D reconstruction problem is ill posed—there are two very different solutions, each of which is feasible. So, when you look at it you alternately see one then the other—you can feel it pop in, or pop out. Without a unique solution your brain flips between the possible solutions. If we had a bit more context it would lock in one interpretation and keep it there. To create this class of optical illusion you must carefully balance two different visual interpretations so closely that they compete.

Similarly, the experiments under which priming shows up are also highly artificial contexts—that is not a criticism because to make a controlled experiment you need this. Part of that is asking people questions about things that they don't know—to which there is no solution.

If you asked a park ranger from Redwood National Park, I don't think he'd be influenced by priming up front—because he gives tours to tourists all the time quoting the world's tallest redwood (by the way, it is a tree named Hyperion that is 379.1 feet tall).

My guess is that the largest priming effect is going to occur when you have somebody who either has no clue about the real answer, or who is grasping for a poorly remembered solution, or sitting on the fence between two compelling solutions. That is the case where context sensitivity would seem to be most important.

A lot of life is spent sitting on the fence trying to decide between multiple solutions or courses of action (purchase decisions, for example), and a lot of life is about things that we don't have a clue about or only dimly grasp. So priming could be quite important even if it is restricted to that case. Or maybe it isn't restricted, and we are all at sea in a storm of contextual metaphors.

Anyway to sum up here are my questions:

1. Is priming strongest when your the answer is indeterminate (i.e. subject doesn't know, or is struggling to get the answer, or to choose between answers)?

2. Or, does priming influence people even on topics where they feel that they know the answer?

3. How important is priming? Is there a way to measure this? Is it a curiosity that occurs in some corner cases (which is how I would characterize optical illusions)?

Finally, I will admit to wondering about more thing, which I am sure you've all be thinking also : why on earth am I arguing with the guy who won the Nobel prize in this area?

Nathan, lots of good questions here, except the last one. Being with you for 36 hours was enough to not be surprised that you are still pushing the debate! Danny is really the right one to have this debate with but I am not sure your questions have good answers. For example, we have no idea what percentage of the time we are fooled by optical illusions. Presumably the answer is a small number, but how would we know?

Let me respond briefly to two of your points regarding my stuff.

First, I agree that unit pricing was a bust, though interestingly, in an experiment done by Jay Russo, when prices were listed in order of unit prices, this had a big effect on behavior. However, I am not excited about that result because we don't know whether it leads to a good outcome if the products are not of homogeneous quality. For example, I pointed out in an old paper (1985) that the price of dish washing liquid per dish washed was inversely related to the price per ounce. So the "most expensive" brands were actually the best values, at least as evaluated by Consumer Reports.

This is an example of the mapping problem Sendhil and I talked about. If consumers don't know how to map their purchase into utility, then markets can break down. Some firms will make money selling diluted dish washing liquid to consumers who use the wrong heuristic for buying (price per ounce). But they won't make huge amounts of money because it costs more to bottle and distribute diluted product, so competition itself does not solve the problem. Nor does Consumer Reports if most people don't read it. (They re-did the study a few years later and got the same result.) My idea of electronic, machine-readable disclosure will not solve these problems completely either, because not everyone will go on-line, but it would make markets more efficient.

The most interesting question is whether it is necessary for the government to mandate the disclosure. Before I address that let's clarify one point. It may not be necessary that the government be the body that decides what the template is for disclosure.

For example, the travel web sites might determine what the disclosure template would be for hotels and airlines. The only problem with this is if the airlines and hotels capture the travel web sites. However, there can be no debate about whether it is necessary for the government to require that the disclosure be made at all.

Consider the case of mortgages. Let's stipulate that some borrowers took out loans that they did not understand, and that mortgage brokers, especially those marketing to the sub-prime sector, were profiting exactly by not disclosing clearly some of the features of the loan. If I am making money precisely because I do not disclose everything, then I cannot be expected to start disclosing voluntarily. (Suppose the guys who play Three-card Monte on the street had to reveal all their tricks. Who would play?)

I have no problems with Nathan's discussion of illusions and the Necker cube—both appear in early chapters in what I have been writing. Indeed, illusions have something in common with priming effects and with most of mental life: the processes that determine them are not represented in consciousness. As Nathan suspected, my objection to the comparison of priming to illusions related to their importance. Illusions do not play a very important role in the modern study of perception, whereas priming is, I believe, critical to understanding how the mind works.

Anchoring effects work on everybody—including professionals, such as judges in actual courtroom cases, who get demonstrably anchored on numbers that have little more content than random numbers. Anchoring also works quite well in negotiations, where getting your number in first gives an advantage (this is experimental, but confirmed by negotiations experts as a phenomenon observable in significant negotiations).

True, anchoring effects are largest in areas in which there is little experience. If you know how tall the tallest redwood is you won't show an anchoring effect. But not everyone knows everything—although there may be one or two exceptions… Anchoring is not a laboratory curiosity. Nor are framing effects, which are best explained as effects of priming—different frames evoke different associations in memory and different emotions.

I cannot resist quoting a paragraph from a discussion of anchoring:

German judges were shown detailed files on a rape case, then told that someone with no expertise in the law—a freshman computer-science student—had recommended a prison sentence for the rapist. One group of judges saw that the amateur had suggested 12 months in prison; a second group read that he had suggested 34 months. Most of the judges were firmly convinced that the opinion of the computer-science student could not influence their own sentencing decision, but they were wrong. When the judges were asked how long a term they would assign in their own courtroom, those who had seen the amateur advice of a 12-month term recommended 28 months in prison; the judges who had seen the 34-month suggestion concluded that 36 months was appropriate.

Well, I am going to sound like I was primed by the recent X files movie, but Danny, I want to believe!

Nevertheless it is not easy to believe it. Despite your having primed me, I'm just not getting it. So while I believe you that it does work on me, in this particular instance it seems to be failing me. So I have a couple more questions.

Perhaps practicing on me will help your book :-), but don't feel compelled to answer if you don't have the time. Indeed one reply you give would be "wait for the book".

My natural inclination is to go in two directions with this:

1. There must be a catch—meaning something which winds up lessening the importance of priming from what you seem to be saying. Here is a list (I will try to be brief) of objections that come to mind.

a. The effect occurs, but it is proportional to ignorance, or indecision, or making your mind up when you don't have enough information (indeterminate or ill posed problem). So it is a very real effect, but only occurs a portion of the time. And in that portion, we don't know what to do anyway (see coin flip argument below).

b. In particular, the kind of question one asks on a questionnaire tends to be something where the person must either grasp at an answer, or make up a hypothetical—or otherwise are particularly prone to the effect. So this might be much stronger in precisely the cases studied in the experiments. Is it known whether people act this way "for real"? For example, one branch of experimental economists holds that doing experiments with real money (which may itself be priming!) makes things more real than hypothetical questions. Have those experiments been done with priming? I can't help but point out here that your colonoscopy experiment was real—to the extreme! Have similarly real world tests been done here?

c. Other influences may drown out priming in cases of interest. The German judges were asked about a sentence—they weren't presiding over an actual trial where they had heard weeks of testimony and had both prosecution and defense repeatedly try to prime them. Would the freshman opinion count as much after all that context?

d. It may last only a short time, in which case the next priming comes in and overwrites it.

2. Or, I believe. But now explain how to reconcile our intuitive view of life with the strange reality you propose. The naive reaction to taking you seriously is the storm tossed sea of metaphors—can we really be reality? Could unconscious suggestion send us lurching between extremes all the time without our being aware of it? Perhaps, but it deserves explanation. How could it be so hidden? Why aren't we all switching political parties, wives, cars, and other decisions randomly based on some damn number we hear, or points we plot on graph paper? How can we go thorough life so gullible? Most aspects of life do not seem like a totally random walk—yet that is what you seem to be leading us. The incredible constancy and directionality of some aspects of life is hard to reconcile with the notion that we are almost whimsically influenced by cheap metaphors.

I think it would help the uptake of this idea if you gave people some sense that one can reconcile priming with our life experience. Otherwise it becomes easier to reject the whole thing. Here are some attempted reconciliations I have thought of:

a. Dick asked whether we know how often optical illusions come into play in real life. I would argue that the neural processing behind optical illusions happens almost constantly. What is rare is that it produces a paradoxical answer. So, the answer to Dick's question depends on whether you mean "tendency to recognize faces" which occurs constantly, or the "picture that might be an old lady's face, or a young woman's torso" (another classic illusion) which occurs only rarely. So, perhaps priming is just as constant as the processing behind illusions, but its impact is usually small. You won't like this because it seems to de-emphasize priming, but there is a pro-priming way to think of this.

b. Similarly one could make a case that lots of subtle things—like the example of culture you mentioned—could be due to priming. Yet culture, while pervasive at a large scale does not utterly dominate individual life. Indeed most people would agree that there is such a thing as culture, while at the same time arguing that they are their own man (or woman), exercising free will. One could argue that priming is not only responsible for culture, but like culture it is simultaneous palpable in some ways and diffuse in others, subject to ex post facto denial.

c. If indeed priming works best when we don't otherwise know, or have enough information, then it could be a substitute for a coin toss. If you are asked a hypothetical question, or are sitting on the fence on a decision, then what does it matter whether it is a priming effect, or a cosmic ray that pushes you over one way versus another? Instead of tossing the coin the brain takes whatever random thing it was told recently and turns up the gain on that noise to get an answer.

d. The sum of sufficiently many random vectors is arbitrarily close to zero. So, maybe we are totally influenced—slaves to the cheap metaphors of life's contextual poetry. But if there are enough of those influences in enough directions, then overall they tend to balance out. Culture would be a good example of a systematic bias, but many other things would wash out. So maybe German judges just go by the last thing they heard, but there are plenty of integers so overall it would wash out over time.

e. Perhaps it is getting caught that is rare. It is very unusual for German judges to get caught in the act of cribbing their jurisprudence from freshman CS majors. Instead we have plenty of ex post facto rationalization, and lots of other window dressing to explain how we act. So perhaps that is the strange part.

f. In fact life is a random walk in most ways and we just don't see it. I don't believe this, but out of completeness thought I would add it.

Finally, I have one additional question. Have you considered "Bayesian priming"?

Suppose that I am an Econ, not a human, and I am a very good Bayesian. Before making a judgment I recall my priors—the prior distribution has a very strong sway on what I do. Let's assume my prior distribution is accurate. But although I aspire to Econ-hood, I am only human. So if we assume my prior distribution is faulty—for example if the last thing I heard influenced my prior distribution, then I could use a good procedure (Bayes' theorem) and get a bad answer.

The question here would be this—if you do numerical priming experiments (guess tallest redwood after hearing a random number), then one might be able to calculate what people's prior distribution was, and how it changed on hearing the priming number.

The answer might be that people are non-Bayesian, or that they can't do the prior. But it seems like there would be some merit in figuring out what effect the priming number is. If you said the tree was 1 million miles high, or 1 inch high, I bet it would be less priming effect. I would guess that the maximum effect occurs once you are in a zone of plausibility (i.e. has a probability consistent with your naive prior), and that the priming number tends to cause you to update the prior distribution in a systematic way.

2. I do not believe there is anything anyone could do within the law to make me vote for a Republican this November.

So yes, of course there are limits to priming effects and to all forms of influence. My point was not that priming could make a person do anything at all. It was that priming has much more influence than people think it could have. Furthermore, people are generally not aware of having been influenced.

Let me respond to your points in your previous message above.

1a. Not correct. The German judges were not completely malleable. They would not have assigned the same sentence to jaywalking. But under the influence of a prime they added an average of 8 months to the sentence.

1b. Yes, similar world test have been done here. Priming has been shown to have a marked influence on the behavior of people in an ultimatum game with real money. I don't know why this should be surprising. People's behavior in games is affected by sniffing oxytocin, which tends to make them more tolerant of exploitation. Oxytocin does not change their nature completely, but it biases their behavior in predictable ways, just as priming does. Much of priming research has been done with real behavior as a dependent variable. Recall the story of the eyes affecting behavior at the "honesty box" (repeated in Monday's NYT). The "artificial questionnaire/no incentive" gambit does not work here.

1c. The power of anchoring effects is such that if a freshman opinion had been the only number proposed as a sentence, it would very likely have an effect. An anchoring effect occurs whenever you recruit arguments to make sense of something that sounds like a solution to your problem.

1d. Two responses to this. First, decisions and judgments are made at a particular time and in a particular context. What priming effects demonstrate is that random features of the context can have a surprisingly large effect on decisions and judgment. The second response is that some priming-like effects are not short-lived. A defendant's physical attractiveness is a long-term prime that has significant effects. And a culture can provide very frequent reminders of the importance of money, or of the importance of community.

2. As I said above, let us stipulate that no one can make me vote Republican. I did not say that behavior is infinitely malleable, only that it is much, much more malleable than people know. So the fact that people stay with wives and political parties is not a challenge to what I said. But there was a lot of luck and a lot of random priming in the choice of the wife, if not of the party. There is a classic study in which male subjects encountered an attractive woman on a swaying bridge in Vancouver, or just off the bridge. They were much more likely to be smitten if the encounter was on the bridge—they did not know exactly why they had been excited. This is one of many ways in which behavior is moved by forces of which we are not aware.

2a. It really depends on what you mean by "small". I think that 8 months is not small, and many significant anchoring effects with real money are quite large in absolute terms. The effects are not huge—they are within the range of what people would consider unsurprising behavior for themselves. But I may be giving too much ground here: in the post-hypnotic suggestion case, people find themselves doing pretty bizarre things and are not subjectively very surprised. They find a reason for what they observe themselves doing. However, we know that even hypnotic suggestion has bounded effects. Repeating myself, I don't know how to use large or small—I do know that the effects are larger than most people think—otherwise the research would not be interesting and we would not be having this exchange.

2b. See above—who claimed that random priming effects utterly dominate individual life? You insist that the effect of priming is either overwhelming or negligible, but of course it is neither.

2c.d. Tell that to the felon who got sentenced. And there is really no reason to expect priming effects to cancel out. Yet again, you respond to a claim that has not been made, that we are totally influenced by context.

2f. This is much stronger than I believe. Life is not a random walk, but there is more randomness than we see.

In response to your final question, the experiments you suggest have been done. Totally ridiculous numbers will not work, but you can be quite extreme and still get large priming effects. Dan McFadden (Econ Nobel Laureate) and I reported on a study in which one group of subjects made free estimates of a set of quantities, or answered hypothetical open-ended questions about contributions to various causes. Other subjects were anchored by a dichotomous question (the redwood example was from that study). The anchors were set either at the 5th or at the 95th percentile of the distribution of free responses. We could measure the anchoring effect as the proportion of the distance between the anchors that was spanned by the responses of the anchored groups. The results of such studies produce a robust estimate: about 50%. I have called this measure the anchoring index.

Nathan, priming and anchoring do not dominate your life—but they affect your choices and judgments more than you can tell by introspection. You can see your face in the mirror but not the brain behind it, and you can see some of what goes on in your mind, but most of it is hidden.

Danny complains that I am objecting to something he never said. I suppose that may be technically true, but what it tells me is that I must have done a poor job of explaining why I was bringing these points up in the first place. My apologies, let me try again.

Priming as Danny presents it is quite a strange phenomenon:

• omnipresent—happening all the time, all around you.

• impossible to guard against.

• equally hard to detect—in yourself anyway, but also in others (unless you have a control group and can do the statistics, as one does in an experiment).

• very important to understanding human perception.

• also very important in terms of real world impact on thinking and decisions, with large real-world consequences.

I'm pretty sure Danny said each of these, one way or another. Or maybe I was just primed to draw these conclusions myself, but I think they are accurate.

If find that set of characteristics to be fascinating. However, they are also strange, and perhaps a bit alarming if you really take them seriously. It very naturally begs a set of other questions.

That is really the point—the explanation of priming (and particularly those factors above) begs some important questions. When Danny says that I am responding to claims he didn't make, I think he is misunderstands this point. I think that that the claims Danny did make beg the questions I was responding to.

If somebody told me "the sun is green", there are two natural reactions I would have. The first would be to be skeptical and discount the assertion, thinking it is either false, exaggerated or occurs in very weird conditions. The second is to accept it provisionally and say "ok, if the sun is green, help me understand and accept that by explaining further how this could it be that I've lived my whole life thinking the opposite". Even if I want to believe, if I get no answer to this second approach, then I surely will be driven back to skepticism. But hey, maybe that's just me.

When explaining something this strange, it is very helpful to provide preemptive rebuttals to the first line of skeptical theories. That way, even before the first few objections arise you can swat them down. It is perhaps even more important to provide the perspective on "how could it be that I've spent my whole life thinking the opposite". I didn't get either one from the discussion at the Master Class. Of course that may just be me being really dense. The brevity of the discussion is clearly a factor. Danny's forthcoming book probably handles this all brilliantly and I should just shut up and wait for it. But I'm impatient so I sought to clarify them in this response and counter response format.

My previous response cataloged a bunch of possible reactions to priming in each direction. The first was a list of ways of possible objections to discount the effect. The second was a set of possible ways to reconcile the phenomenon of priming with a naive intuitive notion of perception.

Frankly, approach this hasn't been all that successful, so I'll stop. Some of Danny's responses are spot on in providing the extra insight that I am looking for—they either rebut a skeptical point, or they explain that. Mostly however, I seem to have phrased things badly so Danny's responses seem aimed at trying to debate a recalcitrant heathen.

Incidentally, the sun is indeed green! But not usually—it is only green during a rare and very brief phenomenon called the "green flash". The rarity and brevity are why we don't think of it that way.

Let's look at the second question first. There have been hundreds of experiments (thousands?) on the phenomena of priming and anchoring (these days often viewed as a species of priming). A common feature of these experiments is that the participants uniformly do not know what is going on (the few who do become aware are normally removed from the sample) and strenuously deny that the manipulation to which they were exposed (typically, the presentation of some obviously irrelevant stimulus) had any effect on their behavior. This highly robust observation does not explain why people can live their whole life without awareness of these phenomena, but it demonstrates the possibility (indeed, the likelihood) that if there is such a general effect we would not be aware of it. It is in the nature of unconscious effects that we are not conscious of them, and it also a universal and fully justified initial response to doubt their existence, generality and significance. I don't know anyone who was not initially skeptical, including myself. It is the sheer mass of results that I find compelling.

To deal with the first question, let me suggest an interpretation that makes the phenomena less weird. We are all aware that our behavior and our thoughts and feelings are highly context-dependent: none of us is quite the same person at home and in the office, in bed or in the subway. We are used to the context-dependency of our behavior and we have stories that make sense of it—social pressure, norms etc. What we are learning from the priming-anchoring effects is that context-dependency is mediated in part by multiple subtle cues of which we are not necessarily aware. The effect of pictures of eyes on contributions to the honesty box illustrates this. People were barely aware of this contextual cue and had no idea it had a large effect on their behavior—they were responding pretty much as if they were under observation. The example shows that it does not take actual fear of social sanctions to make us behave in a manner that would be appropriate in a truly social context. I believe that if you consider the factors that govern our adjustments to the contexts of our lives, the suggestive effects of primes and anchors should become less mysterious.

The novelty of the recent priming literature is in something that I called "Associative coherence" or "the poetry of priming". The characteristic of our responses to stimuli (I believe I used the word VOMIT as an example) is that they are coherent—the entire associative machinery (including the autonomic and skeletal responses that the machinery controls) seems to be reset for a new context. We are more alert, we are prepared to recognize stimuli that are predictable (in a Bayesian sense) in the new context, we are ready to escape etc. This coherent response makes a great deal of sense in an evolutionary context. The fact that some associations appear bizarrely symbolic (e.g., to notions of distance or reminders of money) makes sense in the context of theories of "embodied cognition", which trace some of the concepts people have to early experiences, e.g., of social and physical distance or of the difference between situations in which money and exchange are or are not relevant).

Finally, let's look at the importance of the phenomena. Perhaps wrongly, I read Nathan as proposing that the effects are either extremely powerful or negligible, and my response was that they are somewhere in the middle. I find it helpful to think of behavior as a choice of values along multiple continua (e.g., of friendliness, wariness, effort, driving speed, etc.). At any one time each of these features of our behavior and mental state can be represented as an equilibrium, which is influenced by multiple forces, some of which are internal (habits, intentions, stored knowledge) others drawn from the context. We are not specifically aware of all these forces (any more than we are fully aware of what determines our choice of speed on a winding road), but they are at work, and priming is one of the ways this comes about.

Nathan, thanks for the skepticism. I learned both from the questions and from the answers. In this context, 'recalcitrant heathen' is not pejorative—it refers to someone who says "you will have to do more to convince me", which is a pretty good description of our exchange.

Well, with this I think we can declare our commentary finished! Danny and I seem to be largely in agreement.
But I can't resist one final point. The strangeness of priming is much worse than simply that we are not aware of it—we also don't seem to find its traces afterward.

Psychology is full of unconscious phenomena. So, for example, I can accept that my eyes may dart around and the pupils contract or dilate, betraying my interest in things. That by itself is strange, but easy to reconcile with intuition because you'd never know it without careful observation (with video cameras or the like). Last year Danny told us of the "peak plus end" rule that says people tend to remember the peak and the last bit of an experience (such as pain). This is fascinating stuff, which ultimately is easy to accept because it is explicitly about what we don't remember.

However, in the case of priming, the unconscious phenomenon leaves an indelible mark on our lives. It's not just about something transitory—it is about lasting decisions, like what we pay for communal tea, or what sentences are given in a court of law. I think that this makes it doubly weird. If indeed we were so open to influence, then wouldn't we see that influence in the records of our deeds? Pupil dilation by German judges is one thing—the sentences they give quite another. If in fact there is a meaningful priming influence on such things, why don't we see life as a random walk? If a judge hired for his or her professional competence at administering justice responds to arbitrary priming, then shouldn't that show up in the judicial record? Wouldn't a statistical analysis show it?

That is why it is not enough (for me, anyway) to say it is a robust empirical fact that experimental subjects are unaware of priming. Yes, that is the first step! It makes me believe! But now tell me why don't we find their record of actual decisions to have that influence? If we were all tugged in different directions by factors beyond our control, then wouldn't evidence of that be recorded? If we are tossed about on a stormy sea of metaphors, why don't we notice, even after the fact?

There are many examples where economists and statisticians have indeed used statistics do show cumulative effects of all sorts of influences that may be unconscious—like correlation of height with income. The more one says that priming is important, the more I would expect it to come out in the statistics—probably as variance after known systematic effects were removed. Priming is so dependent on ephemeral context that it would seem that it would appear as the noise rather than then the signal.

Several of the points in my email were attempting to reconcile this aspect—that our lives seem ordered in retrospect. So, if the effect was small compared to other sources of variance, then you wouldn't see it—it would be hidden by other variability. Or perhaps priming is the primary source of variance. Or maybe priming falls within the same range as other forms of variance so it goes unnoticed. Or... at this stage it is time to stop before I reprise the whole discussion all over again. I think I have some idea of what Danny's answer would be here and I look forward to the fuller explanation in his book.