Confirmation bias

You’ve decided that a coworker is disorganized. When your manager asks how he’s doing, you open the project tracker and look for his overdue tasks. When he misses a deadline, you think “typical”. When he finishes something early, you assume someone else must have helped, and a week later you’ve forgotten about it.

Each step, taken alone, looks reasonable. But the search only looked where disorganization would show up, the missed deadline was read as proof, and the counterexample was explained away and then forgotten. The belief now feels better supported than it was, although nothing that could have challenged it was given a fair chance.

• Seeking confirming evidence. In Peter Wason’s 1960 “2-4-6” task, people were told that the numbers 2, 4, 6 fit a rule and had to discover the rule by proposing other sets of three numbers. People typically proposed only sets that fit their own guess (such as “even numbers going up by two”), were told each one fit, and so got no sign that the guess was wrong. The actual rule was any three increasing numbers, which could only be found by trying a set their guess said would fail.
• The positive test strategy. Joshua Klayman and Young-Won Ha argued that much of what gets called confirmation bias is a general habit of testing cases where you expect the thing you’re looking for to be present. This is a strategy for testing, not a wish for a particular answer (see Limits).
• Interpreting evidence to fit. In a 1979 study, people on both sides of a disputed question read the same two studies, one supporting each side. Each group rated the study that agreed with them as more convincing and better conducted, and found flaws in the other. Researchers call this biased assimilation.
• Remembering selectively. In studies reviewed by Raymond Nickerson, people who were asked to recall a theory they favored and the evidence they’d seen tended to remember the evidence as fitting the theory better than it did.
• Persisting with a first impression. Information that arrives early tends to set a view that later information is read through, so the same facts can lead to different conclusions depending on their order.

• Anchoring effect: one leading explanation of anchoring (the selective accessibility model) holds that people test whether the anchor could be the right answer, which brings anchor-consistent information to mind.
• Halo effect: once a first impression of a person forms, later information about them tends to be read in its light.
• Overconfidence: if the evidence you gathered was gathered to fit, it will feel stronger than it is. Nickerson reviews studies in which having to give reasons against your own answer reduced overconfidence, though it didn’t eliminate it.
• Illusory correlation and stereotypes: noticing and remembering the cases that fit an expected link, and not the ones that don’t.

Daniel Kahneman discusses confirmation bias in the same chapter of Thinking, Fast and Slow as “what you see is all there is”: both describe a mind that builds a coherent story from what is in front of it, and confirmation bias adds that a question like “is this true?” steers which evidence comes to mind in the first place.

The counter-questions are “What would I expect to see if I were wrong, and have I looked there?” and “Would I read this evidence the same way if it pointed the other way?”

There is some evidence that this can be trained. In a 2015 study, a single session of a training game or video reduced confirmation bias on the researchers’ own measures, immediately and about two months later. Whether that carries over to everyday reasoning is less clear.

• It names several things, not one. Searching, interpreting, remembering and persisting are different processes, and some are driven by wanting a belief to be true while others happen with no stake at all. Saying “that’s confirmation bias” often doesn’t say which one is at work.
• Looking for confirming cases is often rational. Klayman and Ha showed that testing cases where you expect your hypothesis to hold is a good strategy under common conditions, in particular when the thing you’re looking for is relatively rare and your hypothesis is about the right size. In those conditions, it’s also the test most likely to turn up a case that proves you wrong. The 2-4-6 task is built so that this strategy fails: the guessed rule sits entirely inside the real one, so every positive test gets a “yes”. The error is a mismatch between strategy and task, not seeking confirmation as such.
• Some resistance to changing your mind is reasonable. Nickerson points out that most beliefs that matter can’t be settled by a single piece of evidence, and that dropping a belief at the first contrary sign would be its own mistake. Confirmation bias is about giving evidence unequal treatment, not about failing to change your mind instantly.
• Not every famous result has held up. In the 1979 study, participants also reported that their views had become more extreme after reading the mixed evidence. That polarization was self-reported, and it has often failed to appear in studies that measured attitudes directly, including a recent set of three experiments that found none (see Backfire effect). Rating agreeable evidence as more convincing is a separate claim from becoming more extreme, and the failure of the second doesn’t undo the first.
• It can explain almost anything after the fact. Like What you see is all there is, it’s a useful lens but weak as evidence. Pointing out that someone’s belief could have been reinforced by confirmation bias doesn’t show that it was, or that the belief is wrong.