Chapter XVI · 32 min

Pioneers

Letters, pots, bells, and three axioms

“The most important questions of life are, for the most part, really only problems of probability.”

Probability did not fall out of the sky as a list of axioms. It was argued into existence, slowly, by people who needed to divide a pot, insure a ship, reduce an observation, or say what “almost always” could possibly mean. This chapter is their workshop, not a hall of statues. For each name we keep one idea, fully spelled, so the history is a second pass through the mathematics you already own.

You do not need to memorise dates. You need to see that every intimidating word in this book — conditional, inverse probability, law of large numbers, Gaussian, Markov, axiom — was once a letter between two people who were stuck.

Pascal and Fermat: dividing an unfinished game (1654)

Two players stake equal money on a match that is first to three games. They are interrupted when the score is 2–1. How should the pot be split? The medieval answer was a shrug, or a split by games already won (2:1), which ignores the fact that the leader is one game from finishing while the trailer needs two.

Pascal and Fermat, writing letters in 1654, counted the remaining equally likely futures. At most two more games decide it. The four possible sequences — AA, AB, BA, BB — are interchangeable if the players are evenly matched. In three of them A reaches three wins; only BB gives the match to B. So A should take three quarters of the pot.

That is classical probability, invented to be fair: list the remaining world, treat its atoms as interchangeable, take a ratio. Fermat counted sequences; Pascal counted with a triangle of coefficients. Same number. The argument is Chapter I of this book, written as a quarrel about money.

Huygens: expectation as a fair price (1657)

Christiaan Huygens wrote the first printed textbook of the subject, De ratiociniis in ludo aleae. His primitive notion is not probability but expectation: the fair price of a game. If a lottery pays 10 with chance one half and 0 otherwise, a fair ticket costs 5. That is Chapter VII, before anyone had named a random variable. Huygens’s “value of the chance” is the balance point of this book’s seesaw.

Jacob Bernoulli: the law, slowly (1713)

Ars Conjectandi appeared eight years after Bernoulli’s death. The title is the programme: an art of conjecture — of measuring a chance you cannot see by watching frequencies you can. His theorem, the first law of large numbers, says: toss a coin with heads-chance $p$; let $S_n$ be the number of heads in $n$ tosses. Then, for any strip around $p$,

Read it in English, twice. The running fraction of heads becomes unlikely to sit far from $p$. It does not say the fraction equals $p$. It does not say the next toss gets fairer. It does not pay your gambling debts. It says the sample mean is a shy animal that, given enough room, hides near the true chance. Chapter XI is Bernoulli’s theorem with a modern proof and a warning about what it refuses to claim.

De Moivre and Gauss: the bell as a limit, then as an error (1738, 1809)

Abraham de Moivre, in the second edition of The Doctrine of Chances, showed that a binomial count, properly centred and scaled, looks like a smooth hump — the curve we now call normal. He was approximating $P(S_n=k)$ for large $n$ so that a person with a table could avoid expanding $(p+q{)}^n$ by hand. The central limit theorem of Chapter XI is de Moivre’s hump, grown up.

Carl Friedrich Gauss met the same curve as the law of errors. Repeated measurements of a star’s position scatter. Gauss asked: which density makes the arithmetic mean the most plausible guess? The answer is the Gaussian. Least squares (Chapter X of Stranger in LA) and the normal (Chapter IX here) were twins in his Theoria motus. An astronomer, not a gambler, named the curve most people now meet first.

A tiny picture of de Moivre

A fair coin, $n=4$ tosses. Counts of heads: 0,1,2,3,4 with chances 1,4,6,4,1 over 16. Already a hump: six ways to get two heads, one way to get none. Let $n$ grow and the staircase, scaled, sits under a bell. That is all the mystery. The formula with $e^{-x^2/2}$ is the bell’s name; the idea is “counts pile in the middle.”

Bayes and Laplace: inverting a conditional (1763, 1812)

Thomas Bayes’s essay, published by Richard Price two years after Bayes died, asked: given how often an event has happened, what should we believe about the chance that governs it? In our letters that is

Chapter IV is that inversion, taught with a clinic instead of a billiard table. Bayes’s own example was a ball thrown on a table, and the data were “to the left / to the right of a second ball.” The theology in Price’s cover letter is optional; the inversion is not.

Pierre-Simon Laplace made the inversion a method of science. His Théorie analytique des probabilités (1812) is the first great treatise: generating functions, the rule of succession, the principle of indifference, inverse probability applied to births, tribunals, and the mass of Saturn. When we say “a prior,” we are speaking Laplace’s dialect, whether or not we like the word.

Markov: tomorrow asks only about today (1913)

A. A. Markov wanted to show that the law of large numbers does not need independence. He counted vowels and consonants in Pushkin’s Eugene Onegin, modelled the next letter as depending on the letter just written, and proved that averages can still settle. Chapter XV is that idea: a table of “from → to,” an eigenvector of 1 as the long-run weather. Dependence is allowed, so long as it is short-term.

Kolmogorov: three rules, a whole subject (1933)

Andrey Kolmogorov’s little book Grundbegriffe der Wahrscheinlichkeitsrechnungdid what Hilbert had asked of geometry: name the few properties that make probability a mathematical object, then build. Non-negativity, $P(\Omega )=1$, and countable additivity over disjoint events. Chapter II of this book is that list, written for a reader who has just counted a die. Measure theory is the adult language; the three rules are the contract. After 1933, a quarrel about chance could be a quarrel about a measure, which is to say it could be mathematics.

Shannon: surprise, counted (1948)

Claude Shannon’s “A Mathematical Theory of Communication” named a bit as a unit of surprise. An event of chance $p$ surprises you by $-{\log }_{2}p$ bits. Average surprise is entropy. Chapter XIII is that accounting, used wherever a model is scored — weather offices, compression, and, among other trades, learning. Shannon was an engineer of telephone lines. The vocabulary escaped the wires.

A few more chairs at the table

• Poisson (1837): rare events, a limit of the binomial, the distribution of Chapter VIII that counts mishaps.
• Chebyshev (1867): a crude inequality that already proves Bernoulli’s law — variance as a fence.
• Fisher (1920s): likelihood as a score for a parameter; the estimator as a random variable. Chapter XII and XIV.
• Neyman and Pearson (1933): tests as a two-door policy, with error rates you choose before the data. A different temperament from Bayes, still a way to act under chance.
• Lévy, Lindeberg, Feller: the central limit theorem as a theorem, not a rumour. Feller’s Volume I is the desert-island book of the further-reading chapter.