Chapter V · 22 min

Independence

When information changes nothing

“The concept of independence is the most important in probability theory; it distinguishes this theory from the rest of mathematics.”
— Mark Kac, Statistical Independence in Probability, Analysis and Number Theory (1959)

Two events are independent when learning one does not change the chance of the other. That is a statement about information, not about overlap. Coins flipped on opposite tables are independent and can both be heads. “Even” and “odd” on one die are as dependent as events get: they cannot happen together.

The definition

Events $A$ and $B$ are independent if

P (A \cap B) = P (A) P (B) .

If $P (B) > 0$ this is equivalent to $P (A ∣ B) = P (A)$ : the extra information $B$ was free. The product form is kinder, because it still makes sense when a probability is zero.

A table you can fill with a pencil

Ω = {HH, HT, TH, TT}, each 1/4. Write a 2×2 of first-coin × second-coin:

HH: A and B both happen. Chance 1/4.
HT: A happens, B does not. Chance 1/4.
TH: A does not, B does. Chance 1/4.
TT: neither. Chance 1/4.

The row “first is H” has total 1/2. Inside that row, half the mass is “second is H”. So $P (B ∣ A) = 1/2 = P (B)$ . That is independence as a picture: every row of the table is a scaled copy of the column margins. If one row were (0.4, 0.1) instead of (0.25, 0.25), learning the row would change the chance of the column.

Independent is not disjoint

If $A$ and $B$ are disjoint and both have positive probability, then $P (A \cap B) = 0 \neq = P (A) P (B)$ , so they are not independent. Disjoint events are informative: if $A$ happened, $B$ did not. Independence is the opposite mood. Mixing the two words is the most common vocabulary error in a first course — and a cousin of a different mix-up in Stranger in LA (Stranger in LA), where “independent” means “not a scaled copy of each other.” Same English, two subjects. With the same Venn that tricked us in Chapter III, reread slowly.

Left: disjoint disks. The overlap is empty, so each event is a hard no for the other — that is dependence. Right: independent events are allowed to overlap; the overlap just happens to have area P(A)P(B).

Several events

Pairwise independence is not enough for a crowd. Events $A_{1}, \dots, A_{n}$ are mutually independent when every subcollection factors: for any distinct indices,

P (A_{i_{1}} \cap \dots \cap A_{i_{k}}) = P (A_{i_{1}}) \dots P (A_{i_{k}}) .

The usual counter-example is two independent fair bits $X, Y$ and their parity $Z = X \oplus Y$ . Any two of ${X = 1}$ , ${Y = 1}$ , ${Z = 1}$ are independent; all three are not, because the third is a function of the first two.

Independence is a modelling assumption you put in, or a relation you check.
It is preserved by complements: if A and B are independent, so are A and Bᶜ.
Independent experiments multiply. The sample space becomes a product, and $P$ becomes a product measure.

History

The product that made chance its own subject

Independence is the idea that makes probability a subject of its own, not a chapter of combinatorics. Jacob Bernoulli used independent trials throughout Ars Conjectandi (1713) without isolating the definition. De Moivre multiplied chances of successive tosses as a matter of course. The product rule P(A ∩ B) = P(A)P(B) as a definition, and the warning that pairwise is not mutual, are nineteenth- and twentieth-century tidyings — Chebyshev, Markov, Bernstein, then Kolmogorov’s axioms, which make independence a modelling extra rather than a consequence. Mark Kac’s 1959 line in the epigraph is the moral: without independence, chance is just finite additivity with extra steps. The surprise that disjoint is the opposite of independent has been ambushing students ever since Venn diagrams entered the classroom.

Foundations studio: make the idea yours

This extended studio deliberately slows the pace. It is for a first-time learner who wants to recognize the idea in a new story, not merely reproduce a formula. Work with pencil and paper. Predict before calculating; redraw the pictures; and finish every numerical answer with a sentence in ordinary language.

A mental map before more algebra

Independence is a factorization statement, not a feeling of unrelatedness. It can be checked algebraically and it behaves subtly when more than two events are involved.

A working map for Independence. Cover the labels and reconstruct the chain from memory.

Do not treat the arrows as a theorem. They are a study aid. A strong probability habit is to move back one box whenever a formula feels unmotivated: ask what the experiment is, what information is available, and what quantity the question actually requests.

Three formulas worth being able to narrate

ma t hbb P (A c a pB) = ma t hbb P (A) ma t hbb P (B)

Read this line from left to right and explain what every symbol refers to in the experiment. If a symbol has no story, the model is not finished.

ma t hbb P (A mi d B) = ma t hbb P (A)

Now read the statement backwards: what would have to be known to use it? Backwards reading is often the difference between recognizing a formula and knowing when it applies.

ma t hbb P (X, Y) = ma t hbb P (X) ma t hbb P (Y) q u a d t e x t f or in d e p e n d e n t v a r iab l es

Test the expression at an edge case or simple symmetric case. Probability formulas should survive sanity checks before you trust the arithmetic built on them.

Worked example ladder

A small experiment you can actually do

A five-step habit for every example in this chapter.

What usually goes wrong

When you notice this mistake, do not merely correct the final number. Return to the first line where the model became ambiguous. Probability errors are often representation errors wearing arithmetic clothing.

Questions beginners are right to ask

Does zero correlation imply independence?

Not in general. Independence implies zero covariance when moments exist, but the converse needs special assumptions such as joint normality.

Is independence an observed fact?

Usually it is a modeling assumption tested or justified approximately, not something guaranteed by the labels of variables.

Can conditional independence differ from ordinary independence?

Yes. Two variables can become independent after conditioning on a common cause, or become dependent after conditioning on a common effect.

Where the abstraction earns its keep

For each application, ask what counts as an outcome, what the model treats as random, and which assumptions are approximations. This is how probability becomes a modelling language instead of a catalogue of formulas.

Connections: do not store chapters in separate boxes

Problem-solving clinic: from recognition to fluency

There is a stage where every worked example looks clear but a fresh problem still feels foreign. The cure is not another formula; it is practice choosing the representation. Before equations, do a sixty-second scan: identify the experiment, what is known, what remains uncertain, the quantity being asked for, the assumption doing the heavy lifting, and one impossible answer that gives you a sanity bound.

The expert loop returns every calculation to the original story.

Case clinic A: Two coin tosses

The first toss and second toss are independent under the fair independent-toss model: knowing the first result leaves the second at probability 1/2.

Case clinic B: Pairwise but not mutual

Let two fair bits determine a third by XOR. Each pair is independent, but knowing any two determines the third. Pairwise independence is weaker than mutual independence.

Solve or reason about it twice: once exactly and once with a rough estimate, simulation, or symmetry argument. If the two approaches disagree dramatically, investigate before trusting the more sophisticated calculation.

Debug a confident wrong answer

Two questions to answer without notes

Does zero correlation imply independence? Not in general. Independence implies zero covariance when moments exist, but the converse needs special assumptions such as joint normality.

Is independence an observed fact? Usually it is a modeling assumption tested or justified approximately, not something guaranteed by the labels of variables.

A notebook protocol for proficiency

Give this chapter one notebook page divided into four quadrants: picture, formula, example, mistake. Redraw the main visual from memory, narrate one formula in English, invent a fresh story using the same mathematics, and record the most tempting wrong move. Revisit the page after two days and again after a week.

A mastery check before you move on

Try these without looking back. If one item feels slippery, return to the corresponding example and rebuild it rather than memorizing the answer.

Give a one-minute explanation of the chapter title to a curious teenager without a formula.
Invent a tiny example with at most six elementary outcomes and solve it completely by enumeration.
State one assumption that would make your example invalid and identify exactly which line would break.
Draw the mental map from memory and connect at least two boxes to an earlier or later chapter.
Write one question whose answer you still do not know. Good questions show that the concept has become active rather than passive.