Chapter III · 28 min

Conditional Probability

The Venn lens is not the conditional

“Chance is only the measure of our ignorance.”

Information shrinks the sample space. You rolled a die, and someone tells you the face is even. The six-point world has become a three-point world. Conditional probability is the classical rule, replayed on the world that remains.

The formula is a fraction. The picture people reach for is a Venn diagram. That picture is useful — and it is where the most popular mistake in the whole subject is born: pointing at the lens $A\cap B$ and calling it $P(A\mid B)$. Slow down. The lens is the numerator. The whole of $B$ is the denominator. Conditioning is a ratio inside $B$, not a region of $\Omega$.

The Venn-diagram trap

Draw two overlapping disks, $A$ and $B$, inside a rectangle $\Omega$. The eye is pulled to the vesica — the lens where they overlap. That lens is the event $A\cap B$. Its probability $P(A\cap B)$ is “both happen”, as a fraction of the original world. Conditional probability asks a different question: both happen, as a fraction of the world in which $B$ has already happened.

Suppose, just to put numbers on the disks, that $P(B)=0.40$ and $P(A\cap B)=0.10$. Then

The overlap is one tenth of $\Omega$. Given $B$, it is one quarter of the remaining world. If you report 0.10, you have answered “how often do both happen?” when you were asked “how often does $A$ happen, among the times $B$ happens?”. Those are not the same sentence.

Restricting Ω, with a die you can count

If $P(B)>0$, the probability of $A$ given $B$ is

Read the right-hand side as a ratio of sizes. The denominator is the weight of the new universe. The numerator is the weight of the part of $A$ that still lives there. In the equally-likely finite case this is again a ratio of counts: $|A\cap B|/|B|$.

Two readings of the same fraction

• Restrict. Throw away every outcome outside $B$. Renormalise so the remainder sums to one. Then read off $A$.
• Divide. Keep $\Omega$ as it was. Compute the two masses $P(A\cap B)$ and $P(B)$, and take their ratio.

Same number. The first reading is why the Venn lens is not enough. The second is why the definition is a fraction, not a region.

The multiplication rule

Rearranging the definition gives a way to build joint events from stages:

This is the trap, run in reverse, and it is honest: the lens equals “size of $B$ times the conditional”. For three events, $P(A\cap B\cap C)=P(A)P(B\mid A)P(C\mid A\cap B)$, and so on. This is how a story with a plot becomes a number: write the plot as a chain of conditions, and multiply.

Trees

A probability tree is the multiplication rule drawn. Each edge carries a conditional probability given everything upstream. The probability of a leaf is the product along its path. The probability of an event is the sum of the leaves that belong to it. Leaves are joints. Later edges are conditionals. The first split is a prior. That vocabulary will be the whole of the next chapter.

The law of total probability

If $B_1,\dots ,B_k$ partition $\Omega$ — disjoint events whose union is everything — then any event $A$ can be assembled by cases:

You do not need to see $A$ directly. You need to see it in each branch, and know how likely the branches are. This is the “average the conditionals” theorem, and it is how mixed populations, mixed weather, and mixture models all work. In Venn language: slice $\Omega$ into disjoint disks, measure $A$ inside each, and reweight by how large the disks are.