Some years ago, I started taking house-dance classes. (House is a specific style of dance that goes well with house music.) For more than two decades, I’d been dancing at raves and parties, where people complimented me and sometimes even asked if I was a professional. While I didn’t know what to expect at the studio, I had reason to think I might not be terrible. I immediately found myself the dunce in the back of the class. I hadn’t known what I didn’t know.
My experience demonstrated what’s known as the Dunning-Kruger effect, an often-cited and still-debated concept in both psychology and popular culture.
In 1999, David Dunning, a psychologist then at Cornell University, and Justin Kruger, a Cornell graduate student, published a seminal paper called “Unskilled and Unaware of It.” The paper described four experiments testing college students on humor, logic, and grammar. Dunning and Kruger also asked participants to guess their scores, in both absolute and relative terms. The key finding that kept popping up: Those in the bottom quartile of performance wildly overestimated their skill in that domain (in some studies rating their percentile, on average, around 60 when it was around 10). A smaller finding: Those in the top quartile often underestimated their relative skill (though to a lesser extent).
Dunning and Kruger theorized that the skills needed to perform overlapped with those needed to judge performance, and so poor performers lacked the metacognition to see their failures. Combine that with a universal “better-than-average effect,” in which people overestimate their abilities compared to other people, and the worst performers overestimate the most. Meanwhile, high performers also think they’re better than average but underappreciate the extent to which they are, because of the “curse of knowledge,” in which we think our knowledge is universally shared.
It’s not often that an academic paper finds its way into mainstream culture, but when it does, it’s often misunderstood. I recently contacted Dunning, now at the University of Michigan, to talk about the effect’s legacy. He still vigorously defends the idea, while exploring new manifestations, and has even turned the term into an adjective. As Dunning says, we are all “Dunning-Kruger” at some point in our lives.
What was the origin of your 1999 paper?
Well, the research focus I’ve had off and on is: How well do people know themselves? And just through personal observation, of students in my office, faculty meetings, watching C-SPAN, I would notice that people would say—how should I say it diplomatically?—outrageous things. They have to know what they’re saying can’t be true. I’ve had students come into my office to graciously allow me the chance to concede that the answers I designated as right on the test were wrong.
In his first year of graduate school, Justin Kruger said he wanted to work with me. I said: How much do people know that they’re getting something wrong? And so, we did these studies to see how much people who were performing very poorly on tests were aware of it. We tested logic, grammar, and whether people could spot funny jokes? The first data set that came in was a revelation. I was absolutely astonished. Of course, people are going to overestimate themselves, but we didn’t realize it was going to be at that magnitude.
Others had found that ignorance leads to overconfidence—in physics, chess, tennis, reading, driving, socializing. What was the new piece here?
I think the new piece was putting numbers to it, and putting the formal psychological analysis to it, this distinction between cognition when making a judgment and metacognition when judging the judgment.
[...]
What’s a striking example of Dunning-Kruger today?
AI is the most interesting. The first obvious question is: Do chatbots experience Dunning-Kruger? And the answer is: Oh, yeah. Except for some exceptions, they don’t know when to hold back due to uncertainty. There was a game recently where you make up a phrase and ask a chatbot what it means, and it would give you an answer instead of saying it didn’t know. The other question is: Do chatbots give people Dunning-Kruger? One group found it reversed Dunning-Kruger and made the best performers even more overconfident than the worst people. There’s also work showing that people are very Dunning-Kruger in terms of their ability to tell fake news from real news.
In other work, one professor allowed students to buy insurance if they felt they were going to be in the bottom 50 percent of the class, but very few did. So, misestimations lead to behaviors that don’t work very well. Others have shown that people who are most likely to dismiss scientific consensus are those who know the least science but have the most confidence in their knowledge of the science.
Do you think AI sycophancy, in which chatbots flatter their users, might heighten the Dunning-Kruger effect?
Yes. One of the reasons why people are overconfident in general is that they tend to look for reasons supporting their ideas. In fact, one of the pieces of advice we always give is: If the decision is important, stop and ask why you might be wrong. In planning, one of the procedures that’s recommended for a group is to project yourself into the future and imagine your initiative failed spectacularly. What happened to make it fail? And say, “OK, what can we do to prevent those stories from happening?” That’s called a “pre-mortem.”
Doctors do this as a matter of course. They have a different name for it. They don’t diagnose you; they do a “differential diagnosis.” They may have an idea, but they have to think about what else it can be. OK, you might have Lyme disease, but what else are your symptoms consistent with? We’re going to test you to rule out everything that we can. That’s core to medicine, thinking of alternatives, thinking how you could be wrong.
[...]
How can we check our blind spots?
The best way to do it is with other people. Now, they have to be brave, and they have to tell you that you’re wrong. And I wish we were more skilled at being able to tell each other when we have blind spots.
Is overconfidence ever good?
Confidence isn’t bad. Confidence is something to be managed, and there are times you really want to be confident. In preparation, you don’t want to be overconfident. You want to be, if anything, underconfident, and overprepare. If you’re going into battle, or if you’re going to the Olympic Games, you definitely won’t just go, “Nah, I won’t practice today. I won’t train. I’m going to get the gold.” But on the day that you’re executing, you do want to be supremely confident, because it’ll have a payoff.
When should we most worry about the Dunning-Kruger effect?
When there’s a lot of overlap between performance and judgment. In logic, they’re practically the same thing, but not in sports. I know I’m a horrible golf player. I can tell because the trees tell me when my ball travels to the right and lands in the poison ivy. Also beware lack of exposure to outside models. I thought I was a pretty good cello player, until I heard a recording by Jacqueline du Pré, and I went, “I didn’t know you could do that with this instrument.”
