Scope insensitivity

Three versions of the same appeal

A wildlife group mails a survey to three neighborhoods about covering waste ponds that trap migrating ducks. The versions are identical except for the number of ducks the covers would save each year: 500, 5,000 or 50,000. In each neighborhood, the typical household says it would pay about $30 a year.

No single household did anything obviously wrong; each saw only one number. The claimed bias shows up across the groups: a hundredfold difference in ducks saved produced almost no difference in value. The answers seem to express how people feel about saving ducks in general rather than about saving that many. This mirrors the best-known study of scope insensitivity, although, as Evidence explains, how to interpret that study is itself disputed.

One park, or all of them

Asked what she would pay each year to restore the eroded trails in her county’s largest park, a resident says $40. Asked instead about restoring the trails in all 30 of the county’s parks, which she also visits, she says $45.

The larger program includes the park she valued at $40, plus 29 more she uses. Unless the others are worth almost nothing to her, $5 is too little for them. This is the embedding form: a good is valued about the same on its own as when it is part of a much larger bundle.

The better percentage

A foundation’s grants committee compares two proposals. One would find homes for 80 of the 100 cats at a small shelter. The other would find homes for 200 of the 1,000 cats at a large one, at the same cost. The committee favors the first: “Eighty percent is a real success.”

Two hundred cats is more than eighty, and the cats are what the grant is for. The percentage feels better, so the smaller number wins. This is the less obvious form, which researchers describe as proportional reasoning: judging by the share of a problem solved rather than the amount of good done. The committee might have other reasons (the small shelter could close without help), but “80% is a success” isn’t one of them.

• Scope insensitivity in valuation: willingness to pay stays nearly flat as the quantity grows (the first example).
• Embedding, also studied as part-whole bias: a good is valued about the same alone as when included in a larger bundle (the second example).
• Psychophysical numbing: each additional unit counts for less as the total grows, so the same number of lives or animals saved seems less valuable when more are at risk. Paul Slovic and colleagues named it, and tie it to proportional reasoning (the third example): the share saved carries more weight than the number saved.

• When more of the good really adds less. Value doesn’t have to grow in proportion to quantity. If the first few hundred acres already protect a bird population, more acres may add little of what you care about.
• When your budget, not the need, sets the amount. Someone who gives a fixed sum each year can reasonably give the same amount to a larger cause. What should respond to scope is where the money goes, not necessarily how much there is.
• When you doubt the larger program will deliver. A promise to save ten times as many animals may also be less likely to succeed, and a smaller valuation for it can reflect that.
• When the larger number doesn’t describe a larger outcome. More animals at risk isn’t more animals saved, and a bigger program on paper may overlap with protections that already exist.

The test: if the number were ten times larger or smaller, should my answer change, and does it?

Enough marsh for the herons

A town votes on buying marshland where a colony of herons nests. Biologists report that 300 acres would protect the whole colony and its feeding grounds; a 900-acre option adds dry upland the herons don’t use. Residents support spending almost the same on either.

The larger option is three times the land, but not three times the value to people who care about the herons, since the first 300 acres secure what they want to protect. That’s the more adds less condition. It would be scope insensitivity if the extra acres held a second colony and the vote still didn’t budge.

The fixed giving budget

A retiree gives $300 a year to animal welfare. This year, one appeal describes a program that will treat 100 injured wild birds; another says the same money will treat 1,000. She still gives $300 in total, but gives it all to the second.

The size of her donation didn’t grow with the need, which on the surface looks like the survey pattern. But her total is set by her budget, and where the money goes is highly responsive to scope: she chose the program that helps ten times as many birds. That’s the budget condition.

Daniel Kahneman groups scope neglect with Base rate neglect as extension neglect: judgments that should depend on a quantity (how many birds, how common a category) are made instead from what a typical case is like, which he describes as substituting an average for a sum. What a typical rescued bird is like doesn’t change with the number of birds, so the valuation doesn’t either. He adds that when people see several quantities side by side, the number does have an effect, but a slight and roughly additive one, a pattern he describes as compatible with anchoring and adjustment: the intuitive answer serves as the anchor, and the quantity nudges it only a little.

Christopher Hsee and Yuval Rottenstreich propose that it depends on how people value. When they value by feeling, they respond to whether something is present at all, and much less to how much of it there is; when they value by calculation, value tracks the quantity more steadily. Kahneman and Knetsch’s earlier account was that paying for a public good buys moral satisfaction, which is much the same for a small contribution to a good cause as for a large one.

Critics of the survey evidence offer a different explanation for part of it: people may doubt that a larger program will work, answer an open-ended question about a good they’ve never priced with a round figure, or reasonably value extra units less. On this view, some apparent insensitivity is a measurement problem, not a feature of how people value things.

• Calculate rather than react. In Hsee and Rottenstreich’s experiments, valuations made by calculation tracked quantity more closely than valuations made by feeling.
• Ask what the answer is per unit. Dividing a valuation by the number of animals, acres or people helped shows immediately whether a much larger number was given much the same value, and turns “80%” back into 80 cats. This is practical advice, not a tested intervention.
• Compare the options side by side, but don’t stop there. Kahneman notes that when people evaluate different quantities together rather than separately, the numbers do have an effect, though a modest one. As the grants committee shows, seeing both options isn’t enough if a percentage carries more feeling than a count.

Status: contested. Striking cases of insensitivity to scope are well documented, and psychologists have described mechanisms that would produce it. But the size and generality of the effect are seriously disputed, especially in contingent valuation, the survey method economists use to put dollar values on things like wildlife and parks. Reviews of the same body of studies reach opposite conclusions, and a 12-author guidance paper on best practice notes exactly that. The review for this entry found no preregistered or multi-lab replication of the classic demonstrations.

Foundational work.

• Desvousges and colleagues (1993) gave three separate samples a survey about covering waste-oil holding ponds with wire netting to prevent migratory waterfowl from dying in them. The versions were identical except for the number of birds saved each year: 2,000, 20,000 or 200,000. Asked the most their household would agree to pay each year in higher prices, respondents (roughly 280 to 290 per version) gave mean answers of $80, $78 and $88, with a median of $25 in all three, and the differences were not statistically significant. The study was part of a debate over using such surveys to assess damages after an oil spill; its lead author later wrote that it figured prominently in a 1993 expert panel’s requirement that these surveys pass a “scope test”. Critics pointed to its inexpensive survey mode, open-ended question and small samples, and reanalyses argued that data from this line of research pass a scope test under some conditions. The questionnaire also described all three death tolls as small shares of the flyway’s 8.5 million waterfowl (“much less than 1 percent”, “less than 1 percent” and “about 2 percent”).
• Kahneman and Knetsch (1992) reported the embedding effect: willingness to pay for a good was no different when it was valued alone and when it was valued as part of a larger bundle. In an earlier survey Kahneman described, Toronto residents would pay almost as much to clean up the lakes in one small region of Ontario as all the lakes in the province. Critics argued that the study’s design fell short of good survey practice.
• Fetherstonhaugh and colleagues (1997) documented psychophysical numbing. In one study, respondents said how many lives a medical research program would need to save to deserve a $10 million grant. The median was 9,000 lives when 15,000 were at risk, but 100,000 lives when 290,000 were at risk, so saving 9,000 lives counted for more in the smaller group.
• Hsee and Rottenstreich (2004) found that when people valued by feeling, their valuations were nearly a step function of scope (sensitive to whether any was present, largely insensitive to how much), and when they valued by calculation, closer to a linear function.
• Kahneman’s Nobel lecture (2002) used the birds figures as a leading example of extension neglect, while noting that complete neglect is an “unreasonably strict” test and that people show some sensitivity to quantity in within-person designs.

The dispute over survey evidence.

• Desvousges, Mathews and Train (2012) classified 109 contingent valuation studies by their scope tests: 36% passed, 15% failed and 49% had mixed results. They argued that most studies give too little information to tell whether responses change by an adequate amount.
• Hausman (2012) concluded that the method is “hopeless”, identifying embedding and scope problems as probably the most intractable, and argued that respondents often invent their answers on the spot.
• Burrows and colleagues (2017), in a book-length critique of the method edited by Daniel McFadden and Kenneth Train, reviewed 111 studies of environmental goods valued partly or wholly for their existence rather than use and found that more failed a scope test (54%) than passed (46%), with the failure rate higher among more recent studies. Even among passing studies, the response to scope was often small.
• Whitehead (2016) argued that such classifications are overly pessimistic, and that the right question is whether the response to scope is economically plausible. He cited reviews concluding that most contingent valuation studies pass a scope test, and in a small, purposely chosen set of studies he found sensitivity to scope that was statistically above zero and in a plausible range.
• Hjerpe, Hussain and Phillips (2015) meta-analyzed 22 studies of ecosystem conservation and found that values rose with the amount of conservation, at a diminishing rate.
• Lopes and Kipperberg (2020) surveyed willingness to pay to prevent oil spills of four sizes in Arctic Norway. At first, only the difference between the largest and smallest spills was significant, but sensitivity to scope improved after accounting for respondents’ doubts that prevention would work, excluding some respondents based on follow-up questions, taking sample sizes into account and allowing for diminishing value. They conclude that a failed scope test alone isn’t enough to call a survey invalid.
• Johnston and colleagues (2017), in best-practice guidance for these surveys, write that “economists looking at the same evidence have come to opposite conclusions” about scope tests, and that tests of whether sensitivity is large enough rest on assumptions about preferences that go beyond basic choice theory.

What remains uncertain. How much apparent insensitivity reflects how people value outcomes and how much reflects survey design, doubts about delivery or reasonably diminishing value; how large the effect is in decisions with real money, rather than hypothetical survey answers; and what “sensitive enough” should mean when value needn’t grow in proportion to quantity.