Problems & solutions

Solution

Ω has 36 outcomes. A is the first-six row, the second-six column, minus the double-six: 6+6−1=11, so |A|=11. Sums are even when both even or both odd: 3×3+3×3=18, so |B|=18. A ∩ B: among the 11 outcomes with a six, those with even sum are those whose other face is even (the six is even). If the first is 6, second ∈ {2,4,6}; if the second is 6 and the first is not (to avoid double-counting 6,6), first ∈ {2,4}. That is 3+2=5. Thus P(A)=11/36, P(B)=1/2, P(A∩B)=5/36.

Solution

P = C(4,2)C(48,3)/C(52,5). C(4,2)=6, C(48,3)=17,296, C(52,5)=2,598,960, so 6×17296/2598960 = 103776/2598960 ≈ 0.040.

Solution

(a) Conditioning on {BB, BG, GB} leaves three equally likely outcomes, one of which is BB, so 1/3. (b) Conditioning on {BB, BG} (older-first notation) leaves two outcomes, so 1/2. (c) The information specified different subsets of Ω. “At least one” is not a named child; “the older” is. Always write the set before the ratio.

Solution

The two outcomes are not interchangeable: the coin’s construction (or a long frequency) distinguishes them. Classical counting would force 1/2. The right assignment is P({H})=2/3, P({T})=1/3, P(Ω)=1, P(∅)=0. Same sample space, different weights.

Solution

De Morgan: (A ∪ B)ᶜ = Aᶜ ∩ Bᶜ. Complements: P(A ∪ B) = 1 − P((A ∪ B)ᶜ) = 1 − P(Aᶜ ∩ Bᶜ).

Solution

P(A ∪ B)=0.6+0.5−0.3=0.8. P(Aᶜ)=0.4. P(A ∩ Bᶜ)=P(A)−P(A ∩ B)=0.3.

Solution

By inclusion-exclusion, the union is the sum of singles minus pairwise intersections plus the triple. Dropping the (non-negative, once regrouped) intersection corrections yields the inequality; more elementarily, A ∪ B ∪ C is covered by A, B, C so subadditivity of P (itself from additivity plus monotonicity of the leftovers) gives the bound. Equality holds when the three events are pairwise disjoint.

Solution

A={1,2}, B={3,4}. Intersection empty, so P(A ∩ B)=0, while P(A)P(B)=(1/3)(1/3)=1/9 ≠ 0. Disjoint positive-probability events are dependent: each announces that the other failed.

Solution

Even faces {2,4,6}; those >4: {6}; so 1/3. Definition: P({6})/P({2,4,6})=(1/6)/(1/2)=1/3.

Solution

After an ace, 3 aces remain among 51 cards, so 3/51=1/17. Both aces: (4/52)(3/51)=1/221.

Solution

P(R)= (1/2)(2/5)+(1/2)(4/5)=3/5. P(A|R)=[(1/2)(2/5)]/(3/5)=1/3. The red-heavier urn B explains more of the reds.

Solution

The right-hand side is P(A)·[P(A ∩ B)/P(A)]·[P(A ∩ B ∩ C)/P(A ∩ B)] = P(A ∩ B ∩ C), provided P(A)>0 and P(A ∩ B)>0.

Solution

P(A|B)=0.12/0.40=0.30. The 0.12 is the lens A ∩ B, a slice of Ω. A conditional is a ratio: that lens divided by the whole of B. No single region is P(A|B).

Solution

Hearts are red, so H ⊂ R, H ∩ R = H, P(H ∩ R)=13/52=1/4. P(H|R)=(13/52)/(26/52)=1/2. The lens-guesser reports 1/4 (‘how often is it a heart-and-red?’). The question asked was ‘among red cards, how often a heart?’, which is 1/2. Restricting to the 26 red cards makes the 13 hearts half of the new world.

Solution

P(T⁺)=0.99·0.10+0.05·0.90=0.144. Posterior=0.099/0.144=0.6875 ≈ 0.69. Tenfold prevalence roughly fourfolds the posterior — priors matter.

Solution

Positive pile: 8+48=56. Of them 8 have the condition. 8/56=1/7 ≈ 0.14.

Solution

P(loan)=0.40·0.02+0.01·0.98=0.0178. Posterior=0.008/0.0178≈0.45. You treated the appearance of this one word as the whole data — a one-feature naïve Bayes. Other words would multiply extra likelihood factors if assumed independent given the label.

Solution

Posterior scores: 0.1·0.8=0.08 versus 0.6·0.2=0.12, so MAP is H1. MLE looks only at 0.1 vs 0.6 and also picks H1. If the likelihoods were 0.1 and 0.3, MLE would still pick H1 (0.3>0.1) while MAP would pick H0 (0.08>0.06).

Solution

Hypotheses: spam vs ham. Prior of spam: 0.02. Data: the word ‘loan’ appears. Likelihoods: P(loan|spam)=0.40, P(loan|ham)=0.01. Posterior of spam: ≈0.45. The missing piece is P(loan)=0.40·0.02+0.01·0.98=0.0178, the marginal of the data (sometimes called ‘the evidence’). 0.45 is not a likelihood.

Solution

P(kitchen|empty)= (0.10·0.7)/(0.10·0.7+1·0.3)=0.07/0.37≈0.19. 0.7 is the prior, 0.10 is the likelihood of the data given kitchen, 1 is the likelihood of the data given hallway. The empty hook is the data, not a probability.

Solution

English evidence is the observation itself: the positive test, the witness, the dataset. Bayesian ‘evidence’ often means the marginal P(data), which is a probability that normalises the posterior. In the clinic the positive test T⁺ is evidence-in-English; P(T⁺)=0.0594 is evidence-in-jargon. Mixing the two is why people say ‘the evidence is 0.99’ when they mean the likelihood.

Solution

The prior for the second look is 0.19, yesterday’s posterior. P(kitchen|empty, empty)=(0.10·0.19)/(0.10·0.19+1·0.81)=0.019/0.829≈0.023. Two empty looks all but rule the kitchen out. Sequential Bayes is the same theorem; the word ‘prior’ just means ‘belief before this clue’, not ‘belief before any clue ever’.

Solution

P(A∩B)=1/4=P(A)P(B). P(A∩C)=P(HH)=1/4=P(A)P(C). Pairwise A,B,C: also P(B∩C)=1/4. But P(A∩B∩C)=P(HH)=1/4 ≠ 1/8 = P(A)P(B)P(C). Pairwise independent, not mutually.

Solution

P(A ∩ Bᶜ)=P(A)−P(A ∩ B)=P(A)−P(A)P(B)=P(A)(1−P(B))=P(A)P(Bᶜ).

Solution

P(X=(1,1)|Y=1)=0.64, P(X=(1,1)|Y=0)=0.09. Posterior ∝ 0.64·0.5 versus 0.09·0.5, so 0.64/0.73 ≈ 0.877.

Solution

P(A)=1/2, P(B)=1/2, P(A∩B)=P({3})=1/6 ≠ 1/4. Dependent. Overlapping the centre point correlates them.

Solution

Masses (in 36ths): 1,2,3,4,5,6,5,4,3,2,1 for k=2..12. F_X(4)=P(X≤4)=(1+2+3)/36=6/36=1/6.

Solution

Values: (1−3.5)²=6.25 with P=1/6 (from 1 and, wait: 1 and 6 both give 6.25, 2 and 5 give 2.25, 3 and 4 give 0.25). So 6.25 with 1/3, 2.25 with 1/3, 0.25 with 1/3.

Solution

A fair coin mapped H↦1,T↦0; a fair die mapped even↦1, odd↦0. Different Ω, same push-forward on {0,1}.

Solution

Every real is either ≤ a or > a, never both, so the preimages partition Ω. Hence P(X≤a)+P(X>a)=1, i.e. F_X(a)+P(X>a)=1.

Solution

E[X]=3.5. E[X²]=(1+4+9+16+25+36)/6=91/6. Var=91/6 − (7/2)² = 91/6 − 49/4 = (182−147)/12=35/12.

Solution

Let I_j indicate that the j-th ace is in the hand, j=1..4. P(I_j=1)=C(51,4)/C(52,5)=5/52. E[sum]=4·5/52=5/13. Alternatively four indicators for positions in the hand: each card is an ace with chance 4/52, five cards, E=20/52=5/13. Same number.

Solution

E[X]=4/52, E[Y]=4/52 by symmetry, so E[X+Y]=8/52=2/13. They are dependent: P(Y=1|X=1)=3/51 ≠ 4/52. Linearity did not ask.

Solution

Var=35/12, so P≥2.5 ≤ (35/12)/6.25 = (35/12)/(25/4)= (35/12)·(4/25)=7/15≈0.467. Exact: |X−3.5|≥2.5 means X=1 or 6, probability 1/3≈0.333. The bound is true and slack.

Solution

Expectation is not a prediction of a typical face; 3.5 is not on the die. E[X] is the balance point of six equal masses on {1,…,6}, equivalently the long-run average of many rolls. Sum of faces 21, divided by 6, is 3.5.

Solution

E[X]=1·0.8+10·0.2=2.8. The mode is 1, where most mass sits. The rare 10 has leverage: it pulls the fulcrum right of the crowd. E[X²]=1·0.8+100·0.2=20.8. Var=20.8−2.8²=20.8−7.84=12.96.

Solution

C(5,2)(1/3)²(2/3)³ = 10 · 1/9 · 8/27 = 80/243.

Solution

E[trials]=1/p=5. Failures = trials−1, expectation 4. (This is the other geometric convention.)

Solution

Per page p=2/1000=0.002. Binomial: (1−0.002)^1000 ≈ e^{-2} ≈ 0.1353. Poisson with λ=2: P(0)=e^{-2} same approximation. The Poisson is the limit the binomial is already hugging.

Solution

No. Geometric requires i.i.d. trials with constant p. Without replacement the chance of an ace on the next card changes (and the trials are dependent). The waiting time is a function of a hypergeometric-like process; it has a closed form P(X=k)= (48 choose k-1) ways to miss then hit, over permutations, but it is not geometric.

Solution

(a) Bernoulli p=1/6. (b) Binomial n=10, p=1/6. (c) Geometric p=1/6 on {1,2,…}. (d) Hypergeometric N=52, K=4, n=5 — without replacement. (e) Poisson λ=0.3, the rare-event counting model.

Solution

Hypergeometric: P(X=2)= C(4,2)C(48,3)/C(52,5). E[X]=5·4/52=5/13 by linearity (each card is an ace with chance 4/52).

Solution

Length 1.5 over length 4, so 3/8. P(X=2)=0 because a point has no length.

Solution

P(X>t)=e^{-λt}. P(X>10)=e^{-1}≈0.368. P(X>25|X>15)=P(X>10)=e^{-1} by memorylessness. Same number.

Solution

P(X≥130)=P((X−100)/15 ≥ 2)=P(Z≥2)=1−Φ(2)≈1−0.977=0.023.

Solution

∫_0^2 c x dx = c·2 =1 ⇒ c=1/2. E[X]=∫_0^2 x·(x/2) dx = (1/2)(8/3)=4/3.

Solution

A density’s height is not a probability. P(X=0.5)=0 because a point has no length. P(0.4≤X≤0.6)=0.2, the area (length) of the interval. Integrate, do not read off f(0.5).

Solution

(a) Uniform(0,1). (b) Exponential. (c) Normal, by the central-limit story. P(X>μ+2σ)=P(Z>2).

Solution

P(X=0)=0.4, P(X=1)=0.6. P(Y=1|X=1)=0.4/0.6=2/3. Independence would need 0.4=P(X=1)P(Y=1)=0.6·0.7=0.42, which is not 0.4. Dependent, slightly.

Solution

Var(X+Y)=4+9+2=15. Var(X−Y)=4+9−2=11.

Solution

E[X]=0, E[XY]=E[X³]=0 because X³ is odd on a symmetric support (values −1,0,1). So Cov=0. But Y=X², so {Y=1}={X≠0}, complete dependence. Correlation saw no linear trend: Y is a U-shape in X.

Solution

A common cause (heat) can induce association without any mechanism from cones to casualties. A causal claim needs an intervention or a design that blocks the common cause — randomised assignment, an experiment, or a justified causal graph — not a covariance.

Solution

9/n ≤ 0.01 ⇒ n ≥ 900.

Solution

E[S]=50, Var=25, sd=5. P(S≥59.5)≈P(Z≥(59.5−50)/5)=P(Z≥1.9)≈0.029.

Solution

The 11th toss is still fair — independence. The average including the ten heads still converges to 1/2; the ten become negligible in the n-denominator. No compensatory tail is required.

Solution

Bernoulli(0.1) has variance 0.09; mean of 400 has variance 0.09/400=0.000225, sd=0.015. Deviation 0.03 is 2 sd. Chebyshev: P(|dev|≥0.03)≤0.000225/0.0009=0.25. Not surprising by Chebyshev; a CLT estimate would call it about P(|Z|≥2)≈0.05, mildly unusual but not shocking.

Solution

ℓ(p)=k log p + (n−k) log(1−p). Derivative k/p − (n−k)/(1−p)=0 ⇒ k(1−p)=(n−k)p ⇒ p̂=k/n.

Solution

Log-likelihood = const − (1/(2σ²)) Σ (y_i − w·x_i)². Maximising it is minimising the sum of squares. Least squares is Gaussian MLE.

Solution

Posterior ∝ L(w) exp(−λ‖w‖²). Log posterior = log L − λ‖w‖² + const. Maximising is minimising −log L + λ‖w‖².

Solution

P(x|Y=1)=0.7·0.3=0.21, times prior 0.4 → 0.084. P(x|Y=0)=0.2·0.8=0.16, times 0.6 → 0.096. Posterior odds favour Y=0. Predict 0. (Normalising: P(Y=1|x)=0.084/0.180=0.467.)

Solution

Uniform on 6 points: H=log2(6)=ln6/ln2≈2.585 bits. Equivalently −6·(1/6)log2(1/6)=log2 6.

Solution

H=−0.1 log2 0.1 − 0.9 log2 0.9. log2 0.1≈−3.322, log2 0.9≈−0.152. So 0.332+0.137=0.47 bits.

Solution

D=0.8 log2(0.8/0.5)+0.2 log2(0.2/0.5)=0.8 log2(1.6)+0.2 log2(0.4)≈0.8·0.678+0.2·(−1.322)≈0.542−0.264=0.278 bits. Using a fair-coin code on a biased source wastes about 0.28 bits per toss.

Solution

H(p,q)=−∑ p log q. D_KL(p‖q)=∑ p log(p/q)=∑ p log p − ∑ p log q = −H(p)+H(p,q). Rearrange.

Solution

Bias=0−μ=−μ, variance=0, MSE=μ². A degenerate estimator: no variance, possibly huge bias.

Solution

The n-denominator version is the MLE for a Gaussian and is biased downward: E[∑(Xi−X̄)²]= (n−1)σ². Dividing by n−1 removes the bias. The cost is a slightly larger variance; for large n the two agree.

Solution

log p = −½(x−μ)² + const. Score ∂/∂μ = x−μ. I=E[(X−μ)²]=1. Bound: Var(θ̂)≥1/(nI)=1/n. The sample mean attains it.

Solution

The winner was selected by looking at the same data, so its empirical risk is biased downward (you maximised it). An unbiased estimate uses a held-out sample that did not participate in selection. Cross-validation is the same idea, recycled.

Solution

Today π=(0,1). Tomorrow: (0.5, 0.5). Day after: 0.5·(0.7,0.3)+0.5·(0.5,0.5)=(0.60, 0.40).

Solution

Rows must be conditional distributions, so they sum to 1. First row sums to 0.8; second to 1.1. A fair fix of the first is to renormalise: (0.5, 0.5). The second, already noticed, becomes (0.2, 0.8). (Any other repair is a modelling choice; the definition forbids the original table.)

Solution

π_sun·0.3 = π_rain·0.5 and π_rain=1−π_sun give 0.3π_sun=0.5−0.5π_sun, so 0.8π_sun=0.5, π_sun=5/8, π_rain=3/8. Check: (5/8)(0.7)+(3/8)(0.5)=7/16+3/16=10/16=5/8, and the rain slot is 3/8.

Solution

Rows: A: (0,1,0); B: (1/2,0,1/2); C: (0,1,0). Start (1,0,0). After one step: (0,1,0) — certainly B. After two: (1/2, 0, 1/2) — the ends, never B. The chain is periodic: even times on the ends, odd times in the middle. It still has a stationary row (1/4,1/2,1/4), but a single frog does not sit there at a fixed time.