Moving the goalposts

The bus commute

Ana: The bus gets me to work faster than driving.

Ben: I’ll believe that when you show me it beats the car on a Monday morning.

(A week later, Ana shows her Monday times: the bus was eleven minutes faster.)

Ben: Mondays are easy. Try it when it’s raining.

(After a rainy Thursday, the bus is faster again.)

Ben: You probably just caught every green light.

Ben named his own test, and it was met. Instead of saying what he’d learned, he produced a new test, and then an explanation that would dismiss any result at all. Each demand arrives only after the last one has been satisfied, and nothing about the first result gave a reason to doubt it. This is the clear-cut case.

Every request sounds reasonable

A support team trials a new ticketing system. Their manager says, “Show me that response times went down, and we’ll keep it.” The team brings three months of numbers: the median response time fell from nine hours to four. The manager asks whether customers actually noticed. The team runs a survey, and satisfaction scores rose. The manager says surveys are just self-report, and asks for evidence from repeat purchases instead.

Taken one at a time, each request is a fair question: response time, customer perception and customer behavior are all worth knowing about. The problem is the pattern. The first request was put forward as the thing that would settle it, and when it was met, the manager didn’t say “that’s settled, now I’d also like to know X.” He treated it as though nothing had been shown. The tell is that no earlier result is ever banked.

The goalposts move sideways

Jo: If we switch to the cheaper coffee supplier, we’ll save money on the office order.

(A month later, the invoices show a saving of about 30%.)

Lee: Sure, but the coffee isn’t as good.

The new standard isn’t higher; it’s a different question. Whether the coffee is as good may well matter, and Lee is entitled to raise it. The error is using it to act as though Jo’s claim failed. Jo said the switch would save money, and it did. The honest version keeps the two apart: “You were right about the cost. My worry is the taste.”

• Raising the bar: the same kind of evidence, but more of it, or of higher quality, demanded only once the first amount has been supplied. “One study isn’t enough. Show me three.”
• Changing the criterion: switching to a different measure of success once the first is met, as in the coffee example.
• Demanding the impossible: raising the standard until nothing could meet it. When the bar is set where no real evidence could reach it, the move shades into the Nirvana fallacy, rejecting something because it isn’t perfect.
• Lowering your own bar: the mirror image, applied to your own claim. A prediction fails, and what would have counted as its success is quietly redefined so it didn’t. The same metaphor fits; it’s the ad hoc rescue described under Unfalsifiability.

Where the name comes from, and how well it’s studied. The name is a sporting metaphor: moving the goal after the ball is kicked. It’s widely used in everyday argument, but scholarly treatment is thin. It isn’t among the fallacies discussed in the Stanford Encyclopedia of Philosophy‘s entry on fallacies, and it has no heading in Bradley Dowden’s list of 231 fallacy names in the Internet Encyclopedia of Philosophy. In the research literature it mostly turns up in passing: Bjørn Hofmann, analyzing arguments in a debate over a medical technology, mentions moving the goalposts, “also called raising the bar”, without analyzing it further. The closest well-studied idea is the one Karl Popper described in philosophy of science: when a testable theory fails a test, its defenders can always save it by reinterpreting “both the theory and the evidence in order to make them agree”, which he called a “conventionalist stratagem”. Dowden’s list covers the everyday version as an ad hoc rescue: an assumption with no good reason to accept it “other than that it works to save your cherished belief”.

Not the same error: the no true Scotsman. Three rescue moves change three different things. In a No true Scotsman, the group is redefined so a counterexample no longer belongs to it. In moving the goalposts, the standard of evidence changes so evidence that met it no longer counts. In a Motte and bailey, the claim itself retreats to a safer version and then comes back.

Close to special pleading. Demanding more of the other side’s evidence than of your own is a double standard, and a double standard applied to one favored case is Special pleading. Moving the goalposts can happen without any comparison, though: it’s enough that the bar moves after being met.

Asking for more evidence is often exactly right. It isn’t moving the goalposts when:

• The standard was set in advance and is being applied as set. If a higher bar was always the requirement, pointing out that the evidence hasn’t reached it yet moves nothing.
• A flaw in the evidence or the criterion turns up, and is stated openly. Discovering that a test was badly designed is a real reason to change it. What makes it legitimate is that the reason is given, and doesn’t depend on disliking the result.
• The new standard would apply either way. If the evidence had favored your side, would you still demand it? A standard you’d apply in both directions isn’t being moved for convenience.
• The stakes went up. “That’s enough to try it; it isn’t enough to roll it out company-wide” is a higher bar for a bigger decision, and says so.
• What was shown is acknowledged. “Fair enough, that settles the cost question. Before I agree on taste too, I’d want a blind tasting” banks the first result and adds a separate question.

The test: was the earlier standard offered as enough, and is there a reason for the new one other than that the old one was met?

Fixing a test that turned out to be broken

A hiring team says it will judge a take-home coding exercise by whether the code passes the automated tests. Partway through, they discover the tests can be passed by code that simply prints the expected answers. They tell every candidate, explain the problem, add a review of the code itself, and apply it to everyone, including candidates who had already passed.

The criterion changed after some people met it, which is the surface feature of moving the goalposts. But the change was prompted by a flaw found in the test, not by a result the team disliked; it was announced with its reason; and it applies to every candidate equally. That’s the flaw stated openly condition, together with the new standard would apply either way.

The bar was always three gardens

A gardening club tests claims about home remedies the same way every season: a remedy counts as working only if it helps in at least three members’ gardens. One member reports that crushed eggshells doubled her tomato harvest. The others reply, “That’s one garden. Let’s see what happens in two more.”

The member met a standard, but not the club’s, and the club’s standard was stated long before her result came in. Asking for two more gardens raises nothing. That’s the set in advance condition. It would be moving the goalposts if the club had said “try it in your garden and we’ll believe you,” and then asked for more.

People rarely say in advance what would change their mind, and often don’t know until they see the evidence. When a result they didn’t expect arrives, finding a problem with it feels like careful thinking rather than defense. Each new request is usually reasonable on its own, which is why the move is hard to spot in a single exchange: the error is in the sequence, not in any one step.

It also follows the general pull of Confirmation bias: evidence that fits what you already believe tends to get accepted as it is, while evidence that doesn’t gets closer scrutiny. Moving the goalposts is what that uneven scrutiny looks like from the outside, one demand at a time.

• Agree on the standard first. “What would convince you?” asked before the evidence is gathered makes any later change visible, to both of you.
• Bank what’s been shown. “So we agree the bus was faster on Monday and on a rainy day?” Getting earlier results acknowledged keeps them from quietly disappearing.
• Ask whether the new demand would apply to the other side. A standard held only for unwelcome evidence is a double standard.
• Ask whether any evidence would do. If no answer is possible, the disagreement isn’t about evidence, and more evidence won’t settle it.
• In yourself, notice when you reach for a new requirement right after the old one has been met, and ask whether you’d have named it beforehand.