On this page
Most aphorisms are opinions with good rhythm. “What doesn’t kill you makes you stronger” sounds true until you think about anyone who was left weaker. A small group of them are different. They hold up because something underneath them was proved: a theorem, a property of probability, or a piece of logic. The saying is the short version, and the proof has been left out.
This is my collection of those. For each one I give the plain version, what is underneath it, and a source you can check. I’ve left out anything that rests only on a psychology experiment, however famous. Those can be overturned by the next replication. These can’t.
How to read this
Each heading is the saying. The paragraph under it names the result it comes from and links to where that result was published. Where the popular version of a saying is wrong, I say so.
Probability and statistics
“Extraordinary claims require extraordinary evidence.”
This is Bayes’ theorem in one sentence. Your belief after seeing evidence is your belief before it, multiplied by how much more likely the evidence is if the claim is true than if it’s false. When the starting belief is tiny, that ratio has to be enormous before the result moves anywhere near “probably true”. The idea goes back at least to Laplace. Carl Sagan made this wording famous.
“Never be certain of anything you could be wrong about.”
This is Cromwell’s rule, named by the statistician Dennis Lindley. A probability of exactly 0 or 1 can’t be changed by Bayes’ theorem, whatever the evidence, because anything multiplied by zero is still zero. Certainty, in this precise sense, is the refusal to learn. Lindley borrowed the name from Oliver Cromwell’s letter to the Church of Scotland: “I beseech you, in the bowels of Christ, think it possible you may be mistaken.”
“Absence of evidence is (weak) evidence of absence.”
The popular version says the opposite, and it’s wrong. If you looked for something properly and found nothing, the probability that it exists has gone down. How far it drops depends on how likely you were to find it if it were there. A careful search that comes up empty tells you a lot. A glance tells you almost nothing. Either way, the probability can’t go up.
“Before asking how good the test is, ask how common the thing is.”
This is the base rate. A test that is 99% accurate for a condition that affects one person in ten thousand will produce about a hundred false positives for every true one. Most positive results for rare conditions are wrong, even with good tests. Science has the same problem when most tested hypotheses are false to begin with, which is the argument in Ioannidis (2005).
“Extreme results are usually followed by ordinary ones.”
This is regression to the mean, and it needs no psychology at all. Any measurement is part skill and part luck, so an extreme result is likely to include extreme luck, and luck doesn’t repeat on schedule. Praise someone after their best performance and the next one will probably be worse anyway, which teaches managers the wrong lesson about praise. Galton described it in 1886, measuring the heights of parents and children: Galton (1886).
“Look at the planes that didn’t come back.”
This is survivorship bias. During the Second World War, Abraham Wald was asked where to add armour to bombers, using damage patterns on the planes that returned. His answer was to armour the places with no holes, because planes hit there were the ones that never made it back. The full story, from Wald’s declassified memos, is in Mangel and Samaniego (1984).
“With enough chances, the miraculous is guaranteed.”
This is what Diaconis and Mosteller called the law of truly large numbers. If something has a one-in-a-million chance of happening to a given person on a given day, it happens to about ten people in Portugal every day. Somebody, somewhere, will dream of an old friend the night before that friend calls. The surprising thing would be if nobody did. Diaconis and Mosteller (1989).
“Your friends have more friends than you do.”
This is the friendship paradox, and in any network where people have different numbers of friends, it holds on average. Popular people appear in many friend lists, so when you average over friends, they get counted many times. It applies to followers, partners and colleagues too, which makes it a useful thing to remember when scrolling makes everyone else’s life look fuller. Feld (1991).
“The whole can tell the opposite story of every part.”
This is Simpson’s paradox. In 1973, Berkeley’s graduate admissions looked biased against women overall, yet most individual departments admitted women at similar or higher rates. Women had applied more often to the departments that rejected most applicants of either sex. The aggregate and the parts were both correct, and they pointed in opposite directions. Bickel, Hammel and O’Connell (1975).
“If you only see a filtered group, you’ll find links that don’t exist.”
This is Berkson’s paradox. Berkson showed that two diseases unrelated in the population can look related among hospital patients, because having either one makes admission more likely. The everyday version: if you only date people who are attractive or kind (or both), you will notice that the attractive ones seem less kind. The filter created the pattern. Berkson (1946).
Decisions, risk and time
“Avoid ruin above all. The average doesn’t happen to you.”
This is non-ergodicity. A bet can have a positive expected value across many people and still ruin nearly every individual who keeps taking it, because the average across people and the average over one person’s lifetime are different numbers. A repeated bet that loses half your money on a bad round and gains 60% on a good one looks profitable on average and shrinks your wealth over time. Kelly worked out how much to stake so that this doesn’t happen. Peters (2019); Kelly (1956).
“The longer something has lasted, the longer it will probably last.”
This is the Lindy effect, and Richard Gott gave a version of it a formal argument. If you are observing something at a random moment of its life, there’s a 50% chance you are in the middle half, so its remaining life is probably between a third and three times what has already passed. He used it in 1969 on the Berlin Wall, which fell twenty years later, inside his range. Gott (1993).
“Sunk costs aren’t costs. They’re history.”
Only what happens from here on should enter a decision: the costs you will still pay and the benefits you can still get. Money and time already spent are the same whichever option you choose, so they can’t help you choose. This one is pure decision theory. The psychology is only needed to explain why people ignore it.
“Speeding up what isn’t the bottleneck changes nothing.”
This is Amdahl’s law. If a fraction of a job can’t be sped up, that fraction sets a hard limit on the speedup of the whole, however many processors you add to the rest. If 10% is stuck, the best you can ever do is ten times faster. The management version is the Theory of Constraints. Amdahl (1967).
“Work in progress equals throughput times lead time.”
This is Little’s law, and it holds for any stable queue: a coffee shop, a support desk, a team’s backlog. If a team finishes the same number of tasks per week, starting more tasks at once can only make each one take longer. It’s a theorem, not a productivity tip. Little (1961).
Models and measurement
“All models are wrong, but some are useful.”
George Box’s line. A model is useful because it leaves things out, so being wrong in detail is the price of being usable at all. The question is never whether a model is true, only whether it is wrong in ways that matter for the decision in front of you. Box (1976). The older philosophical version is Korzybski’s “the map is not the territory”.
“When a measure becomes a target, it stops being a good measure.”
This is Goodhart’s law, in the wording Marilyn Strathern gave it. Its rigorous cousin in economics is the Lucas critique: a relationship you observed in past data will change once you start making policy based on it, because people respond to the policy. Every metric dashboard eventually teaches this lesson. Lucas (1976).
“There’s no best method for every problem.”
This is the No Free Lunch theorem. Averaged over every possible problem, every optimisation algorithm performs the same. Any method that does better on some problems does worse on others. All real performance comes from assumptions that happen to match the problem you have. Wolpert and Macready (1997).
“Of two equally good explanations, prefer the simpler one.”
This is Occam’s razor, and it has a real justification in Bayesian model comparison. A complex model can explain many different outcomes, so it has to spread its probability thinly across all of them. A simple model that predicts what you actually saw puts more probability on that outcome, and wins. David MacKay explains this in chapter 28 of Information Theory, Inference, and Learning Algorithms.
Logic and philosophy
“You can’t get an ought from an is.”
This is Hume’s guillotine. No amount of data about how things are will tell you how they should be, on its own. Values have to come in from somewhere else. It’s worth remembering whenever a chart is presented as if it settled a decision.
“No voting system is fair to everyone.”
This is Arrow’s impossibility theorem. With three or more options, no method of combining people’s rankings into a group ranking can satisfy a short list of conditions that all seem reasonable. Every voting rule gives up at least one of them. Arrow (1950).
“No hypothesis is ever tested alone.”
This is the Duhem-Quine thesis. When a prediction fails, the theory might be wrong, but so might the instrument, the measurement, or one of the background assumptions you didn’t think of as assumptions. Experiments test bundles, never single claims.
“Before removing a fence, find out why it was put there.”
This is Chesterton’s fence. Read it in probability terms: anything that has survived for a while probably solved a problem, even if you can’t see the problem yet. That doesn’t make it right. It means the burden is on you to find out what it was for before removing it.
“Invert, always invert.”
This is the mathematician Carl Jacobi’s advice, made popular by Charlie Munger. Many problems become easier if you solve the opposite one first. To avoid failure, work out in detail how you would cause it. In a project, this is the pre-mortem: assume it has already failed, and explain why.
Coda: the sides of a story
People say there are three sides to every story: yours, mine and the truth. The rigorous version combines the map and the territory with Duhem-Quine. Every account, including the one that claims to be objective, depends on an instrument: who was watching, what was written down, what was left out. So there’s a fourth side, and it’s the one to ask for first: what was actually measured.
Whatever happens, happens. It just happens with a probability.