Chapter II · 16 min

Events and Axioms

What probability is allowed to be

“The theory of probability as a mathematical discipline can and should be developed from axioms in exactly the same way as geometry and algebra.”
— Andrey Kolmogorov, Foundations of the Theory of Probability (1933)

An event is a yes-or-no question about the outcome. “The die showed even.” “The classifier erred.” “The waiting time exceeded ten seconds.” Each of those picks out a subset of $Ω$ . The algebra of events is the algebra of sets, and the rules of probability are the few constraints that keep that algebra from floating free of the words chance.

The algebra of events

If $A$ and $B$ are events, so are their union $A \cup B$ (either happens), their intersection $A \cap B$ (both happen), and the complement $A^{c}$ (A does not happen). The impossible event is the empty set $\emptyset$ ; the certain event is $Ω$ itself.

Two events inside Ω. The lens is the intersection; the whole shaded region is the union. Complements are whatever remains of the rectangle.

Two events are disjoint (mutually exclusive) when $A \cap B = \emptyset$ . Disjoint is a statement about sets. It is not independence; we will spend a whole chapter making that distinction painful enough to remember.

Kolmogorov’s axioms

A probability is a function $P$ from events to real numbers obeying three laws. For the finite sample spaces of this book’s first half, they read:

Non-negativity: $P (A) \geq 0$ for every event $A$ .
Normalization: $P (Ω) = 1$ .
Additivity: if $A \cap B = \emptyset$ , then $P (A \cup B) = P (A) + P (B)$ .

On infinite spaces the third axiom is strengthened to countable additivity: the probability of a countable pile of disjoint events is the sum of the pile. You do not need that yet. You do need the consequences that already follow from the finite version.

Consequences you will use constantly

Probability is not a mood

The axioms do not tell you what $P$ means. They tell you what any meaning must obey if it is going to share a name with geometry. Frequency in repeated trials, degree of belief, and normalized counting all fit. A number that fails additivity — “I am 70% sure of A and 70% sure of not-A” — is not a probability. It is a pair of moods.