Chapter V · 16 min

Independence

When information changes nothing

“The concept of independence is the most important in probability theory; it distinguishes this theory from the rest of mathematics.”
— Mark Kac, Statistical Independence in Probability, Analysis and Number Theory (1959)

Two events are independent when learning one does not change the chance of the other. That is a statement about information, not about overlap. Coins flipped on opposite tables are independent and can both be heads. “Even” and “odd” on one die are as dependent as events get: they cannot happen together.

The definition

Events $A$ and $B$ are independent if

P (A \cap B) = P (A) P (B) .

If $P (B) > 0$ this is equivalent to $P (A ∣ B) = P (A)$ : the extra information $B$ was free. The product form is kinder, because it still makes sense when a probability is zero.

Independent is not disjoint

If $A$ and $B$ are disjoint and both have positive probability, then $P (A \cap B) = 0 \neq = P (A) P (B)$ , so they are not independent. Disjoint events are informative: if $A$ happened, $B$ did not. Independence is the opposite mood. Mixing the two words is the most common vocabulary error in a first course, and it is worth a slow reread — with the same Venn that tricked us in Chapter III.

Left: disjoint disks. The overlap is empty, so each event is a hard no for the other — that is dependence. Right: independent events are allowed to overlap; the overlap just happens to have area P(A)P(B).

Several events

Pairwise independence is not enough for a crowd. Events $A_{1}, \dots, A_{n}$ are mutually independent when every subcollection factors: for any distinct indices,

P (A_{i_{1}} \cap \dots \cap A_{i_{k}}) = P (A_{i_{1}}) \dots P (A_{i_{k}}) .

The usual counter-example is two independent fair bits $X, Y$ and their parity $Z = X \oplus Y$ . Any two of ${X = 1}$ , ${Y = 1}$ , ${Z = 1}$ are independent; all three are not, because the third is a function of the first two.

Independence is a modelling assumption you put in, or a relation you check.
It is preserved by complements: if A and B are independent, so are A and Bᶜ.
Independent experiments multiply. The sample space becomes a product, and $P$ becomes a product measure.