One Response to “Dan Kahneman on the Situation of Intuition”

Robertsaid

I was quite surprised to see Kahneman opine in Foregin Policy that Nassim Nicholas Taleb should be considered one of the world’s top intellectuals. Also strange is that Kahneman is often referenced in articles about both Taleb and the financial catastrophe.

Why? In the speech shown here, Kahneman expounds on the benefits of statistical prediction and endorses automated forecasting, particularly for picking stocks. Taleb’s black swan book is essentially a scathing indictment of how quantitative forecasting can go off the rails, and he rightly blames it for much of the current crisis.

If Kahneman’s research can be used to vaguely explain both the case for automated prediction and the complete failure of automated prediction, it’s too elastic to be useful.

Regarding the hot hand, Adam Alter and Danny Oppenheimer suggest disentangling when Kahneman-Tversky heuristics lead to contradictory results, seemingly by doing more laboratory studies and developing additional heuristics that are slightly less vague. At best, this line of inquiry might prolong the day of reckoning.

Responding to Gigerenzer’s critique, Kahneman downplays the concern that his heuristics are no more useful than crystal balls, since “much good psychology would fail this criterion.” For those of us who aren’t psychologists, the question has been begged.

The problem relates to a more general one concerning external validity, and where increased skepticism is warranted. In disciplines like medicine, the validity problem is in determining whether a sample reflects a population. In a clinicial trial of a drug, we may wonder whether results apply to minorities if they weren’t participants in the trial. Psychology also has this difficulty, as samples are usually college students. But there’s a more serious issue, which would only arise in medicine if a trial for one drug was the basis for approving another.

When explaining human behavior in an applied problem, a psychologist sets up an analogy between the situation under scrutiny and a situation previously studied in the laboratory. For example, question of interest: do drivers change lanes too much? Possible lab result: grocery shoppers don’t change lanes often enough. While psychology is an empirical science and studies are analyzed quantitatively, the analogy formation process is an imprecise task in qualitative reasoning, and statistical inferences from the lab can be misleading. Quoting statistician David Freedman out of context, this qualitative analogy formation process “may well stand on [its] own rather than being subsumed under the systematic data collection and modeling activities” traditionally considered methodology. In my view, most “good psychology” involves designing experiments where the analogy is credible for an important problem. The trouble with Kahneman’s heuristics is that people can use them to justify whatever outcome they want.