Illusory correlation

The knee that knows the weather

“My knee always aches before it rains. I can feel a storm coming a day ahead.”

The rainy days that followed an aching knee are memorable, because they seem to confirm something. The aches followed by sunshine, and the rain that arrived with no ache at all, don’t register as evidence of anything and fade. To know whether the knee predicts rain, you’d need to count all four kinds of day. When researchers tracked 18 arthritis patients’ pain against the weather for more than a year, they found no significant association between the pain and the weather conditions each patient had named.

The small team that “always” slips

A company has a large team of 30 on one floor and a small team of 10 on another. Over a year, both miss about one deadline in ten. By December, several managers agree that the small team “is the one that misses deadlines”.

The rate is the same, so there’s nothing to find. But missed deadlines are the rare, noticeable events, and the small team is the less familiar group, so the pairing of the two stands out and gets remembered and repeated. This is the less obvious form, where no prior expectation is needed: the link is manufactured by which combinations are distinctive. It’s the pattern studied in laboratory experiments with made-up “Group A” and “Group B” (see Evidence).

Pasta before a race

A runner has eaten pasta the night before her best races for years and believes it makes a big difference. Her training log shows that her times after a pasta dinner are, on average, only a few seconds faster, and several of her best races followed other meals.

Here the correlation may be real, but it is far weaker than she thinks. The great races after pasta became part of her story; the great races after curry and the slow ones after pasta didn’t. An illusory correlation doesn’t have to be pure invention: overestimating the strength of a real link is the same error. Concluding that the pasta causes the good races would be a further step (see Cum hoc ergo propter hoc).

• Expectancy-based: the link people expect is the one they “see”. In the classic clinical studies, people shown diagnoses alongside drawings attributed to hypothetical patients reported that the signs they expected to go with a diagnosis (such as unusual eyes with suspiciousness) had appeared together often, even when they hadn’t.
• Distinctiveness-based: two infrequent things (a small group and a rare behavior) seem to go together because rare pairings stand out, as with the small team above.
• Illusion of causality: the closely related belief that an action produces an outcome that in fact happens just as often without it, as with a remedy for an illness that usually clears up on its own. See also Illusion of control.
• Patterns in randomness: perceiving a link between sequences that are unrelated, one ingredient in the belief in a hot hand.

• When all the combinations were counted. A log that records the times one thing happened without the other, and the times neither happened, can support a real link.
• When someone else measured it properly. A link established by systematic data, not by remembered cases, is reasonable to rely on even if you’ve only noticed it anecdotally.
• When the data are few and the belief is held loosely. With very little evidence, a mild, provisional lean is not unreasonable, and some researchers argue that part of the classic laboratory effect is a sensible inference from small samples (see Evidence). The error is treating the lean as a finding.
• When a plausible mechanism makes a link worth checking. Expecting a link can be a good hypothesis; it becomes an illusory correlation when the expectation is reported as the observation.

The test: have you counted the times one happened without the other, and the times neither happened?

The gardener’s notebook

A gardener suspects her tomatoes do better in the beds that get afternoon sun. For two summers she records the yield of every bed, sunny and shaded, good year and bad, and finds the sunny beds consistently produce about a third more.

This is a belief about a correlation that started from a hunch, which is where illusory correlations begin. But she recorded all the combinations, including the sunny beds that did badly and the shaded beds that did well, so the link rests on the full table rather than on the memorable harvests.

Dry, windy days

Someone with a diagnosed grass-pollen allergy says their symptoms are worse on dry, windy days in early summer, and checks the local pollen forecast before planning a day outdoors.

It sounds like “my knee knows the weather”. The difference is that the link doesn’t rest on remembered bad days: pollen is measured daily, the allergy was confirmed by testing, and there is a known mechanism connecting the weather, the pollen count and the symptoms.

Tversky and Kahneman proposed an explanation through the Availability heuristic: people judge how often two things occurred together by how easily the pairing comes to mind, and pairings that are strongly associated or distinctive come to mind easily. Research on judging relationships from data finds that people give special weight to the cases where both things are present, which are also the cases that seem to confirm a suspicion (see Confirmation bias). The other three kinds of case are less vivid and feel less like evidence.

For the distinctiveness-based version, researchers disagree about the mechanism. The original account is that rare pairings are better remembered. Others propose that people learn less precisely about the smaller group, simply because they’ve seen fewer examples of it; that judgments reflect incomplete learning, which fades with more experience; or that the effect arises in how people express what they’ve learned rather than in what they learned.

• Draw the table. Write down the four combinations and estimate, or better, count each one. The empty cells are usually where the belief falls apart.
• Ask how often the outcome happens anyway. A remedy that “works” for an illness that usually passes on its own, or a ritual before a game your team usually wins, needs a comparison with the times it wasn’t used.
• Keep a record before deciding, not after. A log kept from the start captures the unremarkable cases that memory drops.

There is some tested support for teaching the first two. In Spain, a class-period workshop on base rates, control conditions and confounding, compared with classes that hadn’t yet had it, reduced causal illusions in a task given to 1,668 high school students in 40 schools, with effects still present in a smaller follow-up six months later. That research measured the illusion of causality rather than illusory correlation as such.

Status: replicates robustly. A meta-analysis found the distinctiveness-based effect reliable, and four large preregistered experiments found it again in people’s evaluations of the groups. Honest caveats remain: the effect in those experiments was small, it didn’t always show up in estimates of how often things happened, and researchers still debate its mechanism and, for one version, whether it is an error at all.

The foundational studies. Loren Chapman introduced the term in a 1967 study of word pairs, and with Jean Chapman applied it to clinical judgment. As Tversky and Kahneman summarized the clinical studies, naive judges were shown diagnoses together with drawings supposedly made by patients, and later markedly overestimated how often natural associates (such as suspiciousness and peculiar eyes) had appeared together. They largely “rediscovered” popular but unfounded clinical lore, the illusion persisted even when the actual correlation was negative, and it kept judges from noticing relationships that really were present. In the distinctiveness-based paradigm introduced by David Hamilton and Robert Gifford in 1976, participants read about members of two made-up groups, one twice the size of the other, performing desirable or undesirable behaviors in the same proportion in both groups. As later researchers describe it, participants overestimated how often the smaller group had behaved undesirably and rated it less favorably. (This entry’s descriptions of the Chapmans’ and Hamilton and Gifford’s papers are based on those later accounts.)

Replication and review.

• Mullen and Johnson (1990) meta-analyzed the distinctiveness-based research and found the basic effect “highly significant, and of moderate strength”, stronger when the rare behavior was negative and when more examples were presented. The analysis predates modern corrections for publication bias. (Based on the abstract.)
• Van Dessel and colleagues (2021) ran four preregistered online experiments, each analyzing about 1,260 to 1,540 participants, using made-up groups. In all four, how people rated the groups showed the illusory-correlation pattern, with small effects that didn’t appear in every condition. Participants’ estimates of the proportion of positive statements about each group did not show it in the first two experiments, and in the other two showed it only when negative statements were the more common kind. None of the implicit (automatic) measures showed it.
• Weigl, Mecklinger and Rosburg (2018) found illusory correlations even when positive and negative descriptions were equally frequent, a condition in which their simulations of one leading explanation predicted none, and the effect did not decrease over time. Weigl and colleagues (2020) replicated the effect again after extended learning.
• Redelmeier and Tversky (1996), besides the arthritis patients, found that 97 college students tended to perceive correlations between sequences that were in fact uncorrelated. (Based on the abstract.)

What is disputed.

• The mechanism. Weigl and colleagues (2020) describe the evidence for the original “shared distinctiveness” memory account as heterogeneous, and their own results did not support it. Murphy and colleagues (2011) found the effect absent early in learning, present after moderate amounts, and eliminated with more experience, and argued that it reflects incomplete learning rather than distinctiveness or information loss.
• Whether it’s irrational. Costello and Watts (2019) argued that, given only a smaller sample from one group and a larger sample from another, associating the rare feature with the smaller group follows from a standard rule of probability (the Rule of Succession), so the distinctiveness-based pattern is not the error it has been taken to be. (Based on the abstract.) This argument does not apply to the expectancy-based version, where people report co-occurrences that contradict the data in front of them.
• Related but separate. The illusion of causality, reviewed by Matute and colleagues (2015), is a closely related research tradition using contingency tasks; its findings bear on illusory correlation but are not the same claim.

What remains uncertain. Why the distinctiveness-based effect occurs; how much of it reflects reasonable inference from small samples; and how large it is outside laboratory tasks. That people often report links between things that the evidence in front of them doesn’t show is well supported.