You probably don’t know what you’re talking about – you only think you do.

August 31, 2013

The observation that people with limited knowledge of a topic believe they know more than they do is popularly known as the Dunning-Kruger effect, named after the psychologists who published the 2009 paper, Unskilled and Unaware of It: “How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments”[1] The paper analyzes and confirms this phenomenon that has been recognized throughout history.

“One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision.”[2] – Bertrand Russell (1872 – 1970)

“The only thing that I know is that I know nothing.” – Socrates (c. 469 BCE – 399 BCE)

The following chart is a construct for understanding how objective assessment of knowledge and self assessment of knowledge are not lineally correlated. The horizontal axis represents increasing objective assessment of knowledge. For example, correctly solving an increasingly complex series of math problems. The vertical axis represents a person’s self assessment of their knowledge, e.g. how good they believe they are in math. In an ideal world, these two variables, self assessment and objective assessment, would be positively correlated in some way as would be represented by the red dashed line. In this idealized case, as the person’s self assessment of their knowledge increased it would be matched by a corresponding increase in actual knowledge.

In fact, there is a positive correlation between self assessment of knowledge and objective knowledge in the first segment of the chart, represented by Segment B. During this phase, as people learn more about a subject their self assessment increases. At Point C, a subset of the population begins to realize they don’t know as much about the subject as they thought and their self assessment of knowledge begins to decline despite objectively knowing more about the subject than they did earlier.

Those that “make it over the hump” tend to be less effusive about sharing their knowledge because they assume their peers have equal or greater knowledge of the subject. In reality, those in Segment D may have far greater knowledge than those in Segment B but a third party observer may believe that the Segment B group knows more simply because they think they know more and are eager to share what they believe is their expansive knowledge.

The self assessment of the Segment D group declines until it reaches a nadir and then begins to increase again. These are the people that will become true authorities in the subject through advanced education and professional peer interaction. This group’s self assessment rises but will never quite reach the level of a group of people at Point C. The Point C people have enough information, vocabulary and ego to have convinced themselves that they are experts whose statements have the same veracity as the true experts and their opinions should be given the same or greater weight.

Rumsfeld’s unknown unknowns

“Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.” —Feb. 12, 2002, Donald Rumsfeld, Department of Defense news briefing[3]

Rumsfeld has received criticism for his statement of knowns and unknows but in the context of Dunning Kruger his statement makes perfect sense. He is in essence saying that there are unknowns of which we are unaware placing him firmly on the right side of the chart. More people should understand the concept of unknown unknowns.

Experts on the internet

One of the best examples of the Dunning Kruger effect is the proliferation of people who read health related documents on the internet and believe that they actually have the same level of understanding as a physician who has studied the subject for years and has treated thousands of patients. This is not to say there isn’t a trove of good information available on the internet. It is just that in many cases the person collecting the information lacks the knowledge or inclination to critically evaluate the validity of that information and lacks a broader context to understand its relevance. It’s like the blind man holding the elephant’s tail and thinking he has a snake. There is a bigger picture and there are unknown unknowns.

Because these people believe or feel they are experts they tend to overlook or diminish information that does not support their particular topic. As an example, this confirmation bias is apparent in the anti-vaccination movement that clings to studies that have been totally dismissed by the medical and scientific community as meaningless. Legitimate studies are rejected by this group because they don’t support their expected premise while anecdotes from internet forums or questionable studies with even more questionable results are cited as evidence. Not all studies are created equal. Some are well designed while others are fatally flawed and produce no meaningful information.

One of the most important thing true scientists do is to distinguish bad studies from the good. It’s not necessarily difficult. One needs only to review the study design, the methods, the background of the authors, the journal publishing the study, the editorial process, the funding source, etc. Even if all of those parameters are credible there is the challenging aspect of placing the study results in context to the greater body of knowledge. One study, however well designed, should never have the power to void all previous research. Remember the elephant’s tail.

Dunning Kruger permeates our daily lives

As you look around it’s easy to see Dunning Kruger in action in our daily lives. As you drive to work you’ll undoubtedly see drivers that are disconnected from safe driving and the rules of the road. Just count the number of times a turn signal isn’t used when it should have been or count how many people stay in their lane when turning from one street to another. My guess is that the number of drivers signaling and turning properly is some number less than 100%. And yet, if you could stop each driver I would speculate that virtually all would classify themselves as an excellent driver. Dunning Kruger

In the business meetings have you ever heard an enthusiastic presentation by a department head and realized you knew far more than she did and yet she was the expert? Dunning Kruger.

Why is Dunning Kruger so prevalent?

Because it works. Confidence persuades, confidence sells but Dunning Kruger implies that competency or knowledge isn’t a requirement for confidence. In some ways it is more efficient to learn just enough about a subject to convince yourself you’re an expert and then begin the process of convincing others. From this perspective there is diminishing value in increasing subject knowledge if you can be the expert in the room with less than others. Dunning Kruger also implies that when the expert is talking there may very well be others in the room with far greater knowledge but are silent because they realize they don’t have all the answers. After the expert finishes talking, it may be wise to find members of the second group and solicit their thoughts and opinions.

[2] The actual quote is “Brute force plays a much larger part in the government of the world than it did before 1914, and what is especially alarming, force tends increasingly to fall into the hands of those who are enemies of civilization. The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. Even those of the intelligent who believe that they have a nostrum are too individualistic to combine with other intelligent men from whom they differ on minor points. This was not always the case.”

This quote was from pages 203 and 204 of Russell’s May 1933 essay “The Triumph of Stupidity” in “Mortals and Others, Volume I: American Essays 1931-1935” (ISBN13: 9780415125857). The context for the essay was the rise of Nazi Germany and was written five months after Hitler was appointed Chancellor of Germany by Paul von Hindenburg, the President of the Weimar Republic. Russell’s lament that minor differences keep intelligent men apart seems not too dissimilar from what has developed in American politics. Is this a case of history not necessarily repeating itself but rhyming?

Thank you for writing this. I have been keenly interested in the subject of competency and its inversely proportional relationship with absolute confidence. I found your article while searching for the exact words of the Bertrand Russell quote.

I took the liberty of reproducing your chart but made sure to note the source and copyright. Please let me know if this should be removed. Thank you.