X0.3.1

Lesson 0.3.1 — Bayes’ Rule and Gaussian Distributions: The Prior, the Likelihood, and the Posterior

Week 4 · Prerequisites: 0.1.1, 0.1.2, 0.2.1, 0.2.2.

Prior results used. This lesson uses 0.1.1’s tensor model (shape, dtype, device, broadcasting), 0.1.2’s .item() / .detach() distinction for metric extraction, and 0.2.1’s SE(3) composition (for the multivariate case where a pose’s translation is the estimated quantity). Each is restated at the point of use.

Hands forward. This lesson hands forward one capability: the learner can write Bayes’ rule for a continuous random variable, identify the prior, the likelihood, and the posterior, apply the Gaussian conjugate update in both precision and covariance form, and compute the posterior’s mean and covariance by hand for a 1D and a 2D problem — the capability that Lesson 0.3.2 (maximum likelihood and MAP) and 0.3.4 (the Kalman filter) assume.

Running case. The same 6-DOF MuJoCo arm from 0.2.1–0.2.4, with a noisy joint encoder on joint 2: the true angle q₂ is unknown, and the encoder reports y = q₂ + ε with ε ~ N(0, σ_y²). A prior on q₂ comes from the arm’s task (the arm is reaching for a target, so q₂ is likely near a nominal value). The posterior p(q₂ | y) is the updated belief, and this lesson’s central result is that the posterior is also a Gaussian with a mean that is a precision-weighted average of the prior mean and the measurement, and a variance that is smaller than either.

§L.0 — Lesson map, objectives, and misconceptions

§L.0.1 — Lesson map

§L.0.2 — Objectives

By the end of this lesson you will be able to do eight things:

1. State the four terms of Bayes’ rule (prior, likelihood, posterior, evidence) and their roles, for both discrete and continuous random variables.
2. Write the univariate Gaussian’s pdf, identify its mean and variance, and compute the pdf at a given point by hand.
3. State the three properties of the univariate Gaussian (closure under linear maps, sum of independent Gaussians, and the 68–95–99.7 rule) and apply each to a robot-encoder problem.
4. Write the multivariate Gaussian’s pdf, interpret its covariance’s eigenvalues and eigenvectors, and state what a non-positive-definite covariance means.
5. State the four operations on multivariate Gaussians (linear map, conditioning, marginalization, sum of independent) and apply each to a 2D example by hand.
6. Derive the Gaussian conjugate update in precision form (1/σ_post² = 1/σ₀² + 1/σ_y², μ_post = σ_post²(μ₀/σ₀² + y/σ_y²)) and compute the posterior’s mean and variance for a specific 1D problem.
7. Distinguish precision (inverse variance) from variance in the update formulas, and diagnose the “posterior variance is larger than either” bug from its symptom.
8. Produce a written “Bayes audit” of an estimation problem: the state, the prior, the likelihood, the posterior, and the units and magnitudes of each variance.

§L.0.3 — Misconception inventory

Each misconception is displaced in the section named.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

This is the ninth lesson. The warm-up draws from 0.2.1 and 0.1.1 — 0.2.2–0.2.4 are one to three steps back and 0.1.4 is five steps back; 0.2.1 is the most relevant prior.

Warm-up 1 (from 0.2.1, SE(3) composition). The arm’s end-effector position is the top-right block of T_B^EE = T_1·…·T_6. Is this position a deterministic function of the joint angles, or a random one? Answer: Deterministic given the joint angles — the FK is exact. It becomes random when the joint angles are random (unknown), and the FK is the map that pushes the angles’ distribution to the end-effector’s distribution. How this is used: §L.3’s “linear map of a Gaussian” is the linearized version of this push-forward, and the lesson’s multivariate example is a 2D linearized FK.

Warm-up 2 (from 0.1.1, tensors and broadcasting). A (3,) mean vector and a (3, 3) covariance matrix are combined with a (3,) observation in (x − μ)ᵀ Σ⁻¹ (x − μ). What is the shape of the result, and which operation is the broadcast? Answer: The result is a scalar (()); (x − μ) is (3,), Σ⁻¹(x − μ) is (3, 3) @ (3,) = (3,), and (x − μ)ᵀ · (3,) is (3,) ᵀ · (3,) = (). The subtraction x − μ is the element-wise op, not the broadcast. How this is used: §L.3’s multivariate pdf is exactly this expression, and the lab’s mvn_pdf must produce a scalar for a single observation.

Warm-up 3 (from 0.2.1, the inverse). For a 3×3 positive-definite matrix Σ, what is Σ⁻¹’s relationship to Σ’s eigenvalues, and what does the relationship say about Σ⁻¹’s definiteness? Answer: If Σ has eigenvalues λ_i > 0, then Σ⁻¹ has eigenvalues 1/λ_i > 0, so Σ⁻¹ is also positive-definite. How this is used: §L.4’s precision form uses Σ⁻¹ (the precision matrix), and its positive-definiteness is what makes the precision form’s “add” formula well-defined.

Warm-up 4 (from 0.1.2, .item()). A learner computes a scalar from a tensor and stores it in a Python list. Why must the learner call .item(), and what is the risk if they do not? Answer: .item() extracts the Python scalar and drops the graph reference; storing the tensor itself retains the graph and consumes memory. The risk is the OOM from device retention. How this is used: §L.5’s lab computes pdf values and stores them in a list; each stored value must be a float, not a tensor.

Part A — Random Variables, pdfs, and Bayes’ Rule

§L.1 — Objects: the four terms and their roles

A random variable X is a quantity whose value is not known with certainty. A probability density function (pdf) p(x) assigns a non-negative value to each possible x, and the density satisfies ∫ p(x) dx = 1. For a discrete random variable, the pdf is a probability mass function and the integral becomes a sum.

Bayes’ rule for two random variables X and Y is:

with the four terms:

1. Prior p(x) — the belief about X before observing Y.
2. Likelihood p$y \mid x$ — the probability of observing Y = y given that X = x. (For continuous Y, this is a density, and its value at a point can exceed 1.)
3. Posterior p$x \mid y$ — the belief about X after observing Y.
4. Evidence p(y) = ∫ p$y \mid x$ p(x) dx — the probability of observing Y = y under the prior. It is a constant with respect to X; it is the normalizer that makes the posterior a density.

The roles:

• The prior and the likelihood are inputs — they are what the learner knows before and what the sensor reports.
• The posterior is the output — the updated belief.
• The evidence is the normalizer — it does not change the posterior’s shape (as a function of x), only its scale, and its value is often not needed if only the posterior’s relative probabilities matter.

The proportional form p$x \mid y$ ∝ p$y \mid x$ p(x) drops the evidence and keeps the shape; it is the form used when the evidence is hard to compute (which is the common case for high-dimensional x) and the normalizer can be recovered by integration or, in the Gaussian case, by completing the square.

§L.1.1 — Result: the prior and the likelihood are inputs; the posterior is the normalized product; the evidence is the normalizer

Claim. For a prior p(x) and a likelihood p(y|x), the posterior p(x|y) = p(y|x)p(x) / p(y) is a density, and it is the only density proportional to p(y|x)p(x). The evidence is p(y) = ∫ p(y|x)p(x)dx, and it is the normalizing constant that makes the posterior integrate to one.

Proof sketch. The product p(y|x)p(x) is non-negative (it is a product of non-negative densities), and its integral over x is p(y) > 0 (assuming y is observed). Dividing by p(y) gives a non-negative function that integrates to p(y)/p(y) = 1, so it is a density. Any other density proportional to p(y|x)p(x) differs by a multiplicative constant, and the only constant that makes it a density is 1/p(y), so the posterior is unique.

Corollary. The posterior’s shape is determined by the product p(y|x)p(x); its scale is determined by the evidence. For a Gaussian prior and likelihood, the product is a Gaussian (Part D), so the posterior is a Gaussian whose mean is a precision-weighted average and whose variance is 1/(1/σ₀² + 1/σ_y²) — smaller than either.

Corollary. The evidence is expensive for a high-dimensional x (the integral is over the full state), and the proportional form is often the only practical form. For Gaussian problems, the evidence is analytically computable (p(y) = N(y; μ₀, σ₀² + $σ_y$²) for the 1D case), and the posterior’s mean and variance are closed-form (Part D).

§L.1.2 — Worked-example ladder

WE-L.1.a — Fully worked: a discrete Bayes update, by hand

Input. A robot’s joint-2 angle q₂ ∈ {0, π/2} (two hypotheses), with a prior p(q₂ = 0) = 0.6, p(q₂ = π/2) = 0.4. A sensor reports y = "high" with likelihoods p(y = high | q₂ = 0) = 0.1, p(y = high | q₂ = π/2) = 0.8.

Trace.

1. Prior: p(q₂ = 0) = 0.6, p(q₂ = π/2) = 0.4.
2. Likelihood: p(y = high | q₂ = 0) = 0.1, p(y = high | q₂ = π/2) = 0.8.
3. Unnormalized posterior:
   • p(y = high | q₂ = 0) · p(q₂ = 0) = 0.1 · 0.6 = 0.06
   • p(y = high | q₂ = π/2) · p(q₂ = π/2) = 0.8 · 0.4 = 0.32
4. Evidence: p(y = high) = 0.06 + 0.32 = 0.38.
5. Posterior:
   • p(q₂ = 0 | y = high) = 0.06 / 0.38 ≈ 0.158
   • p(q₂ = π/2 | y = high) = 0.32 / 0.38 ≈ 0.842
6. Check: 0.158 + 0.842 = 1. ✓

Output and interpretation. The prior favored q₂ = 0 (0.6 vs. 0.4); the measurement’s likelihood favored q₂ = π/2 (0.8 vs. 0.1); the posterior favors q₂ = π/2 (0.842 vs. 0.158) because the likelihood’s evidence for π/2 is strong enough to overcome the prior’s small advantage for 0. The posterior is more confident than either — it puts 84% on the correct hypothesis, versus the prior’s 60% and the likelihood’s ratio of 8:1.

Self-explanation 1. Before reading on: in step 3, the unnormalized posterior is [0.06, 0.32]. What would the posterior be if the prior were [0.5, 0.5]? What if the prior were [0.9, 0.1]?

Answer. With a uniform prior: unnormalized [0.1·0.5, 0.8·0.5] = [0.05, 0.40], evidence 0.45, posterior [0.111, 0.889] — the likelihood dominates. With a [0.9, 0.1] prior: unnormalized [0.09, 0.08], evidence 0.17, posterior [0.529, 0.471] — the prior nearly overcomes the likelihood, and the posterior is close to uniform. The prior’s strength matters when the likelihood is weakly informative; the likelihood dominates when it is strongly informative.

WE-L.1.b — Completion problem

Input. The same robot, but the sensor is less reliable: p(y = high | q₂ = 0) = 0.4, p(y = high | q₂ = π/2) = 0.6. The prior is [0.6, 0.4].

Trace (final 25% as prompts).

1. Unnormalized posterior: ⟨your turn: compute [0.4·0.6, 0.6·0.4]⟩
2. Evidence: ⟨your turn⟩
3. Posterior: ⟨your turn⟩
4. Compare to WE-L.1.a: ⟨your turn: does the posterior still favor q₂ = π/2?⟩

Answers.

1. [0.24, 0.24].
2. 0.24 + 0.24 = 0.48.
3. [0.5, 0.5] — uniform!
4. No — the posterior is uniform, because the prior and the likelihood’s product is symmetric. The prior favors 0 (0.6 vs. 0.4), and the likelihood favors π/2 (0.6 vs. 0.4), and the two cancel exactly.

The lesson. The posterior is not always “the prior plus the evidence”; the two can cancel and give a uniform posterior. The proportional form p(y|x)p(x) is the product, and the product’s balance is what determines the posterior, not the individual factors’ maxima.

WE-L.1.c — Faded: the continuous case

Setup. The prior on q₂ is p(q₂) = N(q₂; 0.5, 0.1²) (a Gaussian centered at 0.5 with std 0.1), and the likelihood is p(y | q₂) = N(y; q₂, 0.2²) (a Gaussian centered at q₂ with std 0.2). The measurement is y = 0.7.

Question. Write Bayes’ rule for p(q₂ | y), and state why the posterior is also Gaussian.

You must produce the intermediate steps. The setup and the answer are given.

Answer. p(q₂ | y) = p(y | q₂) p(q₂) / p(y) = N(y; q₂, 0.2²) · N(q₂; 0.5, 0.1²) / p(y). The product of two Gaussians is a Gaussian (in q₂), because the product’s exponent is a quadratic in q₂, and a Gaussian’s exponent is the unique quadratic form that integrates to a Gaussian. So the posterior is Gaussian, and the division by p(y) only scales it to be a density, not changes its shape.

The lesson. The Gaussian is conjugate to itself: a Gaussian prior and a Gaussian likelihood give a Gaussian posterior. This closure is the subject of Part D, and it is the reason the Gaussian is the default for robot estimation — the update is closed-form.

WE-L.1.d — Contrastive near-miss: omitting the evidence

Correct solution. p(q₂ | y) = p(y|q₂)p(q₂) / p(y), with p(y) computed or the proportional form used with the constant recovered by integration.

Near-miss. p(q₂ | y) ≈ p(y|q₂)p(q₂) — the proportional form, used as if the product is the posterior.

Why the near-miss produces a plausible but incorrect number. For the discrete case, the unnormalized posterior’s entries do not sum to one — WE-L.1.a’s [0.06, 0.32] sums to 0.38, not 1. If the learner treats them as probabilities, the “posterior” has total probability 0.38, and any downstream computation that assumes Σ p = 1 (a decision that uses the posterior, a marginalization that sums over the state, a Bayesian update that uses the posterior as the new prior) is wrong by the missing factor. The error is silent — the numbers look like probabilities (each is between 0 and 1), and the mistake is only visible when the sum is checked.

The discrepancy, stated. The proportional form p(y|x)p(x) and the posterior p(y|x)p(x)/p(y) differ by the scaling 1/p(y), which is a constant with respect to x but not with respect to the sum over x. The proportional form is correct for shape-only reasoning (comparing two hypotheses) and wrong for probability-using reasoning (marginalizing, deciding). The distinction matters whenever the posterior’s absolute values are used, and the fix is to divide by the evidence (or, in the Gaussian case, to complete the square and identify the posterior’s parameters, which implicitly normalizes).

Self-explanation 2. Before reading on: in the discrete case, what is the evidence p(y) for WE-L.1.a, and what would happen if a learner used the unnormalized posterior [0.06, 0.32] as the prior in a second update with the same likelihood?

Answer. The evidence is 0.38. Using [0.06, 0.32] as a prior, a second update with the same likelihood gives unnormalized [0.06·0.1, 0.32·0.8] = [0.006, 0.256], whose sum is 0.262 — a different evidence, but the shape is the same as the first posterior’s, because the likelihood’s shape is the same. The shape is correct (the proportional form preserves it), and the scale is off by the evidence, so the “second posterior” is a scaled version of the first. The fix is to use the normalized posterior as the second prior, so the scale is correct and the second update’s evidence is p(y₂ | y₁) — the predictive distribution’s evidence.

§L.1.3 — Comparative table: the four forms of the update

Design principle (P9.6). Keep the shape and the scale as separate questions, and answer the scale before you use the posterior as a probability. The product gives the shape for free; only the evidence gives the scale. A number that looks like a probability is not one until the scale is right, and the check is a single sum.

§L.1.4 — Concept check

Concept check (displaces M1). A learner says: “Bayes’ rule is p(x|y) = p(y|x)p(x) — the denominator is a detail, and I can drop it.” Which term has the learner failed to model, and what class of bug will that omission produce?

Answer. The learner has failed to model the evidence p(y) = ∫ p(y|x)p(x)dx and its role as the normalizer. The product is non-negative and has the right shape, but it does not integrate to one, so the “posterior” is a density only up to an unknown constant. The bug class is a scale error that is invisible entry by entry: every unnormalized entry lies in [0, 1] — WE-L.1.a’s [0.06, 0.32] — so the numbers pass the only check a learner usually applies. It surfaces when the posterior is used rather than read: a marginalization that sums over x returns 0.38 instead of 1, a decision rule that thresholds the posterior thresholds the wrong quantity, and a second update that takes the unnormalized vector as its prior produces a posterior rescaled by a different evidence. The shape survives all three; the scale does not.

§L.1.5 — Consolidation checkpoint 1

(a) You should now be able to: (i) name the four terms of Bayes’ rule and state which are inputs, which is the output, and which is the normalizer; (ii) say, for any expression of the form p(y|x)p(x) or p(y|x)p(x)/p(y), whether it can be used as a probability, and why; (iii) explain why a Gaussian prior and a Gaussian likelihood give a Gaussian posterior, in terms of the exponent being a quadratic in x.

(b) Transfer probe. The encoder reports a second reading y₂, with p(y₂ | q₂ = 0) = 0.2 and p(y₂ | q₂ = π/2) = 0.7, independent of the first. Starting from WE-L.1.a’s posterior [0.158, 0.842], what is p(q₂ | y₁, y₂) — and what would you have got if you had started from the unnormalized [0.06, 0.32] instead? Answer sketch: The first posterior is the second update’s prior, and it must be normalized before use. Unnormalized: [0.158·0.2, 0.842·0.7] = [0.0316, 0.5894], evidence 0.621, posterior [0.051, 0.949]. Starting from [0.06, 0.32] gives [0.012, 0.224], evidence 0.236, and [0.051, 0.949] again — the same shape, because both inputs were scaled by the first evidence and the ratio survives. The shape is robust to the missing normalizer; the absolute values are not, and the learner who reports [0.012, 0.224] as a probability has a “posterior” summing to 0.236.

(c) Remediation rule. If you cannot do (a)(ii) or (b), return to §L.1.2 and redo WE-L.1.d, then re-attempt (b) without notes.

Part B — The Univariate Gaussian

§L.2 — Objects: the pdf, the parameters, and the three properties

The univariate Gaussian with mean μ and variance σ² has pdf:

The parameters are μ (the mean, the distribution’s center) and σ² (the variance, the distribution’s spread). The standard deviation σ = √σ² has the same units as x, and the variance has the square of x’s units — a fact that matters when combining measurements in different units.

The three properties:

1. 68–95–99.7 rule: P(|x − μ| < σ) ≈ 0.683, P(|x − μ| < 2σ) ≈ 0.954, P(|x − μ| < 3σ) ≈ 0.997. The rule is the quantitative statement of “the Gaussian is concentrated within a few standard deviations of its mean.”
2. Closure under affine maps: if X ~ N(μ, σ²) and Y = aX + b, then Y ~ N(aμ + b, a²σ²). The mean transforms affinely, the variance transforms by the square of the scaling. For a = 1 (a shift), the mean shifts and the variance is unchanged; for b = 0 (a scale), the mean scales and the variance scales by a².
3. Sum of independent Gaussians: if X₁ ~ N(μ₁, σ₁²) and X₂ ~ N(μ₂, σ₂²) are independent, then X₁ + X₂ ~ N(μ₁ + μ₂, σ₁² + σ₂²). The means add, and the variances add — a result that is unique to the Gaussian among the common distributions.

The standardization Z = (X − μ)/σ transforms X to a standard normal Z ~ N(0, 1), whose pdf φ(z) = (1/√(2π)) exp(−z²/2) is the table for the CDF. The standardization is the only calculation the learner needs for the pdf’s values in practice; the pdf’s shape is fixed, and the parameters (μ, σ²) only shift and scale it.

§L.2.1 — Result: the univariate Gaussian is closed under affine maps and sums of independent samples; the two closures are the algebra of the conjugate update

Claim. For X ~ N(μ, σ²), aX + b ~ N(aμ + b, a²σ²), and for independent X₁, X₂, X₁ + X₂ ~ N(μ₁ + μ₂, σ₁² + σ₂²). The two closures are the basis of the Gaussian conjugate update: the prior and the likelihood are combined by treating the observation as a sum (Part D), and the posterior’s parameters are the result of applying the closures.

Proof sketch. For the affine map: the pdf of Y = aX + b is p_Y(y) = p_X((y − b)/a) / |a|, and substituting p_X(x) = N(x; μ, σ²) gives N(y; aμ + b, a²σ²) after completing the square. For the sum: the pdf of X₁ + X₂ is the convolution of p_X₁ and p_X₂, and the convolution of two Gaussians is a Gaussian with means summed and variances summed, which is a direct computation (complete the square in the exponent’s quadratic form).

Corollary. The sum closure is why averaging n independent measurements of the same quantity reduces the variance by n: the average is (1/n) Σ y_i, which has mean μ and variance σ²/n (from the sum’s nσ² and the scale’s (1/n)²). The 1/√n reduction in standard deviation is the central limit theorem’s practical content, and it is the reason a robot’s averaged sensor reading is more precise than a single reading.

Corollary. The affine closure is why standardization works: Z = (X − μ)/σ is aX + b with a = 1/σ, b = −μ/σ, so Z ~ N(0, 1). The standardization is the inverse of the affine map, and it is the only computation needed for a standard normal table.

§L.2.2 — Worked-example ladder

WE-L.2.a — Fully worked: the pdf at a point, by hand

Input. X ~ N(0, 1) (standard normal). Compute p(1.0) and compare to p(0).

Trace.

1. p(0) = (1/√(2π)) exp(−0²/2) = 1/√(2π) ≈ 0.3989.
2. p(1) = (1/√(2π)) exp(−1²/2) = 0.3989 · exp(−0.5) ≈ 0.3989 · 0.6065 ≈ 0.2420.
3. Ratio: p(1)/p(0) = exp(−0.5) ≈ 0.6065 — the density at one std from the mean is about 61% of the density at the mean.
4. Check: P(|Z| < 1) ≈ 0.683 from the 68–95–99.7 rule, which is consistent with the density at ±1 being 0.242 (the area under the curve between −1 and 1 is 0.683).

Output and interpretation. The standard normal’s density at z = 0 is about 0.4, at z = 1 about 0.24, and at z = 2 about 0.054 (0.3989 · exp(−2) ≈ 0.3989 · 0.1353 ≈ 0.054). The decay is exponential in z², so the density at 3σ is 0.0044, and the “tail” beyond 3σ is 0.3% — small, but not zero, and the heavy tail (compared to a uniform or a triangular distribution) is why the Gaussian is the default for sensor noise (which is a sum of many small independent errors).

Self-explanation 3. Before reading on: in step 3, the ratio p(1)/p(0) ≈ 0.61 is not the probability P(|Z| < 1) ≈ 0.68. Why are the two different?

Answer. p(1) is a density (probability per unit z), and p(0.61) is the ratio of two densities. P(|Z| < 1) is an integral of the density over [−1, 1], and its value 0.68 is a probability. The density’s ratio and the probability’s integral are different quantities, and the 68–95–99.7 rule is about the integral, not the density’s value. The confusion between density and probability is one of the most common in Bayesian robotics, and it is the subject of the lab’s break-it test.

WE-L.2.b — Completion problem

Input. X ~ N(2, 4) (mean 2, variance 4, std 2). Compute p(2), p(4), and the ratio p(4)/p(2).

Trace (final 25% as prompts).

1. p(2) = (1/√(2π·4)) exp(−(2−2)²/(2·4)). ⟨your turn: compute⟩
2. p(4) = (1/√(2π·4)) exp(−(4−2)²/(2·4)). ⟨your turn: compute⟩
3. Ratio p(4)/p(2): ⟨your turn⟩
4. Compare to the standard normal’s ratio at z = 1: ⟨your turn: are they equal? Why?⟩

Answers.

1. p(2) = (1/√(8π)) · exp(0) = 1/(2√(2π)) ≈ 0.1995.
2. p(4) = 0.1995 · exp(−4/8) = 0.1995 · exp(−0.5) ≈ 0.1210.
3. p(4)/p(2) = exp(−0.5) ≈ 0.6065.
4. Yes, equal to the standard normal’s ratio at z = 1 (0.6065) — because z = (4−2)/2 = 1, and the standardization makes the ratio p(1σ from mean)/p(0) the same as the standard normal’s. The mean and variance shift the pdf but do not change its shape, so the ratio at a fixed standardized distance is invariant.

The lesson. The Gaussian’s shape is determined by the standardized distance z = (x−μ)/σ, and the pdf’s value at x is (1/σ) φ(z). The scale 1/σ and the shape φ(z) are separable, which is why standardization works and why the 1/σ factor is the only place the variance appears in the pdf’s value.

WE-L.2.c — Faded: the sum of independent Gaussians

Setup. A robot’s joint-2 measurement is the average of n = 9 independent sensor readings, each with X_i ~ N(q₂, σ² = 0.01) (std 0.1). The true q₂ = 0.5.

Question. What is the distribution of the average ȳ = (1/9) Σ y_i? What is the standard deviation of the average? Compare to a single reading’s std.

You must produce the intermediate steps. The setup and the answer are given.

Answer. The sum Σ y_i ~ N(9·0.5, 9·0.01) = N(4.5, 0.09) (from the sum closure). The average ȳ = (1/9) Σ y_i is an affine map with a = 1/9, so ȳ ~ N(0.5, (1/9)² · 0.09) = N(0.5, 0.00111...), and the average’s std is √0.00111 ≈ 0.0333 = 0.1/3. The average’s std is 1/√9 = 1/3 of a single reading’s, which is the 1/√n reduction.

The lesson. Averaging n independent Gaussians reduces the variance by n and the std by √n. This is the central limit theorem’s practical consequence, and it is why robots average multiple sensor readings — the average is more precise than any single reading, by a known factor.

WE-L.2.d — Contrastive near-miss: the density vs. the probability

Correct solution. P(q₂ ∈ [0.4, 0.6] | y) is the integral of the posterior over [0.4, 0.6], and its value is a probability (between 0 and 1).

Near-miss. The learner reports p(q₂ = 0.5 | y) ≈ 3.99 as the “probability that q₂ = 0.5” — the density at the point, which is not a probability and can exceed 1.

Why the near-miss produces a plausible but incorrect number. The density p(q₂ = 0.5 | y) ≈ 3.99 is a legal density value (a Gaussian with std 0.1 has a peak density of 1/(0.1·√(2π)) ≈ 3.99 at its mean), but it is not the probability that q₂ = 0.5 (which is zero for a continuous variable — the probability of any point is zero). The learner who reports “3.99” as a “probability” is confusing the density with the probability, and any downstream decision that treats “3.99” as a probability (e.g., “the probability is 3.99, so it is certain”) is wrong.

The discrepancy, stated. The density p(x) is the probability per unit x, and the probability P(X ∈ [a, b]) = ∫_a^b p(x) dx is the area under the density. The two have different units (per unit x vs. dimensionless), and the density can exceed 1 (which a probability cannot). The peak density of a narrow Gaussian is larger than 1, so the “probability is 3.99” is a unit error, not a probability.

Self-explanation 4. Before reading on: in the near-miss, what is the probability that q₂ ∈ [0.45, 0.55] for a posterior N(0.5, 0.1²)?

Answer. Standardize: z_1 = (0.45 − 0.5)/0.1 = -0.5, z_2 = (0.55 − 0.5)/0.1 = 0.5. P(-0.5 ≤ Z ≤ 0.5) ≈ 0.383 (from the standard normal table, the area between −0.5 and 0.5). So the probability is about 38%, which is a dimensionless number between 0 and 1, and it is not the density 3.99.

§L.2.3 — Comparative table: the four statements about a univariate Gaussian

Design principle (P9.7). Ask which of the four statements a number belongs to before you interpret it. A value like 3.99 is a density, 0.683 is a probability, 0.6065 is a ratio of densities, and 0.0333 is a standard deviation. They are all derived from the same two parameters, so they all “look like” Gaussian output, and the only thing that separates them is the question each answers.

§L.2.4 — Concept check

Concept check (displaces M2). A learner says: “The Gaussian is a mean and a variance — once I have μ and σ² I know everything there is to know, and every other Gaussian fact is just substituting into the pdf.” Which property of the Gaussian has the learner failed to model, and what class of bug will that omission produce?

Answer. The learner has failed to model the closures: the Gaussian is closed under affine maps and under sums of independent samples, and each closure has its own parameter formula that is not a substitution into the pdf. The pdf tells you the density at a point; it tells you nothing about what happens to σ² when you scale a variable by a (it becomes a²σ², not aσ²), or when you add two independent variables (the variances add, not average). The bug class is a plausible but wrong spread: a learner who scales a reading by a and reports the standard deviation as aσ where a < 1 understates the spread, or who averages n readings and expects the variance to fall by √n rather than n. Nothing raises — the numbers are the right order of magnitude and the wrong factor — and the error propagates into every downstream interval, threshold, and covariance that consumes the spread.

§L.2.5 — Consolidation checkpoint 2

(a) You should now be able to: (i) write the univariate pdf and compute its value at a given point by hand, using 1/√(2π) and exp(−½) rather than a library; (ii) state the 68–95–99.7 rule as a statement about integrals, and distinguish it from a statement about density values; (iii) apply both closures — affine and sum — and state which parameter is affected by which operation.

(b) Transfer probe. The averaged joint-2 reading ȳ from WE-L.2.c has std 0.0333 radians, computed from n = 9 readings of std 0.1. The controller consumes the angle in degrees. What is the std of the averaged reading in degrees, and what is the probability that the averaged reading lies within 0.0333 radians of the truth, expressed in degrees? Answer sketch: The conversion is aX with a = 180/π ≈ 57.296, so by the affine closure the std scales by |a|: 0.0333 · 57.296 ≈ 1.91°. The probability is unchanged at 0.683, because standardization makes P(|X − μ| < σ) independent of the units — the interval ±1.91° is the same event as ±0.0333 rad. A learner who scaled the variance by a instead of a² reports a std of 0.0333 · 57.296² ≈ 109°, which is larger than the joint’s whole range and would make the reading useless — a silent unit error, not a raised one.

(c) Remediation rule. If you cannot do (a)(iii) or (b), return to §L.2.2 and redo WE-L.2.c, then re-attempt (b) by writing the conversion as aX + b and asking which closure applies.

Part C — The Multivariate Gaussian

§L.3 — Objects: the pdf, the covariance, and the four operations

The multivariate Gaussian for an n-dimensional vector x with mean μ ∈ ℝⁿ and covariance Σ ∈ ℝⁿˣⁿ (symmetric positive-definite) has pdf:

The covariance Σ has n(n+1)/2 free parameters (symmetric), and its eigen-decomposition Σ = U Λ Uᵀ gives:

• Eigenvalues λ_i — the variances along the eigenvectors’ directions.
• Eigenvectors u_i — the principal axes of the covariance’s ellipsoid.
• The 1σ ellipsoid has semi-axes √λ_i along u_i.

The four operations (which are the multivariate analogues of Part B’s properties):

1. Linear map: X ~ N(μ, Σ), Y = A X + b ⇒ Y ~ N(Aμ + b, A Σ Aᵀ). The mean is affine, and the covariance is sandwiched by A.
2. Conditioning: X = [X_a, X_b] ~ N([μ_a, μ_b], [[Σ_{aa}, Σ_{ab}], [Σ_{ba}, Σ_{bb}]]) ⇒ X_a | X_b = b ~ N(μ_a + Σ_{ab} Σ_{bb}^{-1}(b − μ_b), Σ_{aa} − Σ_{ab} Σ_{bb}^{-1} Σ_{ba}). The conditional mean is linear in b, and the conditional covariance is smaller than Σ_{aa} (the Schur complement).
3. Marginalization: X_a ~ N(μ_a, Σ_{aa}) — the marginal is the block of the joint. Marginalization loses information, so the marginal’s covariance is larger (in the PSD sense) than the conditional’s.
4. Sum of independent: X₁ + X₂ ~ N(μ₁ + μ₂, Σ₁ + Σ₂) for independent X₁, X₂ — the same additivity as the univariate case.

The positive-definiteness of Σ is required: the pdf’s exponent (x−μ)ᵀΣ⁻¹(x−μ) is non-negative only if Σ is positive-definite, and the determinant |Σ| is positive only if Σ is positive-definite. A symmetric matrix that is not PSD is not a covariance, and using it in the pdf gives a negative |Σ| (a complex √|Σ|) or a non-positive exponent (a pdf that is not integrable).

§L.3.1 — Result: the four operations are the multivariate analogues of the univariate properties, and the conditioning’s Schur complement is the source of the “posterior is more certain” result

Claim. For a multivariate Gaussian X ~ N(μ, Σ):

• The conditional X_a | X_b = b has covariance Σ_{aa} − Σ_{ab} Σ_{bb}^{-1} Σ_{ba} (the Schur complement), which is PSD-smaller than Σ_{aa}.
• The marginal X_a has covariance Σ_{aa}, which is PSD-larger than the conditional’s.

The Schur complement’s PSD-smaller-ness is the multivariate statement of “the posterior is more certain than the prior” — observing X_b reduces the uncertainty in X_a by the correlation between them, and the reduction is exactly the Schur complement’s subtracted term.

Proof sketch. The joint pdf’s exponent is a quadratic in (x_a, x_b); conditioning on X_b = b substitutes b and completes the square in x_a, giving the conditional’s mean and covariance. The Schur complement’s PSD-ness is the block matrix version of the Cauchy–Schwarz inequality, and it implies Σ_{aa} − (Σ_{aa} − Σ_{ab}Σ_{bb}^{-1}Σ_{ba}) = Σ_{ab}Σ_{bb}^{-1}Σ_{ba} ⪰ 0, so the conditional is PSD-smaller.

Corollary. The Schur complement is the innovation’s covariance in the Kalman filter (Lesson 0.3.4), and the Kalman gain is Σ_{ab} Σ_{bb}^{-1} — the coefficient of b in the conditional mean. The conditioning’s linear mean μ_a + Σ_{ab} Σ_{bb}^{-1}(b − μ_b) is the linear minimum mean-square estimate of X_a given X_b = b for jointly Gaussian variables, and it is exact (not an approximation) for the Gaussian case.

§L.3.2 — Worked-example ladder

WE-L.3.a — Fully worked: a 2D Gaussian’s pdf and geometry

Input. X = [q₂, q₃] ~ N(μ = [0.5, 0.3], Σ = [[0.04, 0.02], [0.02, 0.09]]). Compute the eigenvalues and eigenvectors of Σ, and interpret the 1σ ellipsoid.

Trace.

1. Σ = [[0.04, 0.02], [0.02, 0.09]].
2. tr(Σ) = 0.04 + 0.09 = 0.13, det(Σ) = 0.04·0.09 − 0.02² = 0.0036 − 0.0004 = 0.0032.
3. Eigenvalues: λ = (tr ± √(tr² − 4 det))/2 = (0.13 ± √(0.0169 − 0.0128))/2 = (0.13 ± √0.0041)/2 = (0.13 ± 0.064)/2. So λ₁ ≈ 0.097, λ₂ ≈ 0.033.
4. Eigenvectors: for λ₁ = 0.097, (Σ − λ₁I) u = 0 ⇒ [[-0.057, 0.02], [0.02, -0.007]] u = 0 ⇒ u₁ ≈ [0.33, 0.94] (normalized). For λ₂ = 0.033, u₂ ≈ [-0.94, 0.33].
5. 1σ ellipsoid’s semi-axes: √λ₁ ≈ 0.31, √λ₂ ≈ 0.18.

Output and interpretation. The 1σ ellipsoid has a long axis of 0.31 along u₁ ≈ (0.33, 0.94) (mostly q₃’s direction) and a short axis of 0.18 along u₂ ≈ (-0.94, 0.33) (mostly q₂’s direction). The correlation ρ = Σ_{12}/(σ₁σ₂) = 0.02/(0.2·0.3) ≈ 0.33 is positive — q₂ and q₃ tend to increase together — and the ellipsoid’s tilt (the long axis is not aligned with either coordinate) is the visual signature of the correlation.

Self-explanation 5. Before reading on: in step 3, the eigenvalues are 0.097 and 0.033, both positive, so Σ is PSD. What would Σ = [[0.04, 0.1], [0.1, 0.09]] imply, and what is the symptom?

Answer. det = 0.04·0.09 − 0.1² = 0.0036 − 0.01 = -0.0064 < 0 — a negative determinant, so Σ is not PSD. The eigenvalues are (0.13 ± √(0.0169 + 0.0256))/2 = (0.13 ± 0.206)/2 ≈ 0.168, -0.038 — one negative eigenvalue. The pdf’s √|Σ| is the square root of a negative number, which is imaginary, and the pdf is not integrable. The symptom is a NaN or a complex value in the pdf, or a negative “probability” — a bug that is loud if the learner checks, but silent if the pdf’s value is used in a downstream computation that does not check.

WE-L.3.b — Completion problem

Input. The same X = [q₂, q₃] ~ N([0.5, 0.3], [[0.04, 0.02], [0.02, 0.09]]). Condition on q₃ = 0.5 (observe X_b = 0.5).

Trace (final 25% as prompts).

1. μ_a = 0.5, μ_b = 0.3, Σ_{aa} = 0.04, Σ_{ab} = 0.02, Σ_{bb} = 0.09.
2. Σ_{ab} Σ_{bb}^{-1} = 0.02 / 0.09 ≈ 0.222. ⟨your turn: verify⟩
3. Conditional mean: μ_a + Σ_{ab} Σ_{bb}^{-1}(b − μ_b) = 0.5 + 0.222·(0.5 − 0.3). ⟨your turn⟩
4. Conditional variance: Σ_{aa} − Σ_{ab} Σ_{bb}^{-1} Σ_{ba} = 0.04 − 0.222·0.02. ⟨your turn⟩

Answers.

1. The conditioning’s mean is 0.5 + 0.222·0.2 = 0.5 + 0.0444 = 0.5444.
2. The conditional variance is 0.04 − 0.222·0.02 = 0.04 − 0.00444 = 0.03556. The conditional std is √0.03556 ≈ 0.189, which is smaller than the marginal std 0.2 — the observation of q₃ reduced q₂’s uncertainty, by the amount of the correlation.

The lesson. The conditional’s variance is smaller than the marginal’s, and the reduction is the Schur complement’s subtracted term Σ_{ab}Σ_{bb}^{-1}Σ_{ba}. The conditioning’s mean shifts toward the observed value, and the shift’s magnitude is the correlation times the innovation (b − μ_b). When the correlation is zero, the mean does not shift and the variance does not reduce — the observation is uninformative about X_a.

WE-L.3.c — Faded: the marginal

Setup. The same X ~ N(μ, Σ). The learner wants the marginal of q₂ (integrate out q₃).

Question. What is the marginal’s mean and variance? Compare the variance to the conditional’s.

You must produce the intermediate steps. The setup and the answer are given.

Answer. The marginal is X_a ~ N(μ_a, Σ_{aa}) = N(0.5, 0.04), with std 0.2. The marginal’s variance (0.04) is larger than the conditional’s (0.03556) — the observation of q₃ reduced the uncertainty in q₂, and the marginal (which does not observe q₃) is less certain. The difference is the Schur complement’s subtracted term, and it is zero when Σ_{ab} = 0 (uncorrelated).

The lesson. Marginalization loses information, so the marginal’s variance is larger than the conditional’s. The ordering (marginal ≥ conditional in the PSD sense) is the multivariate statement of “observation reduces uncertainty”, and it is the reason the Kalman filter’s update reduces the covariance.

WE-L.3.d — Contrastive near-miss: the covariance is Σ not Σ⁻¹

Correct solution. The multivariate Gaussian’s exponent uses Σ⁻¹ (the precision matrix), and the covariance matrix Σ is the inverse of the precision.

Near-miss. The learner uses Σ in the exponent: exp(−½(x−μ)ᵀΣ(x−μ)), which is a different function (not a Gaussian), and the pdf’s normalization is √|Σ⁻¹|, not √|Σ|.

Why the near-miss produces a plausible but incorrect number. The near-miss’s function is bell-shaped (it is a Gaussian with covariance Σ⁻¹, the inverse), and for a covariance close to the identity, the two are close. The learner’s pdf is wrong by the inverse of the intended covariance, so a broad intended covariance becomes a narrow near-miss, and vice versa — the pdf’s width is the inverse of what the learner intended. The error is silent (the function is still integrable, still positive, and still looks like a bell), and it is visible only against the correct pdf.

The discrepancy, stated. Σ (the covariance) and Σ⁻¹ (the precision) are inverses, and using one where the other is intended inverts the distribution’s shape. The convention is Σ⁻¹ in the exponent and √|Σ| in the normalizer, and the notation is easy to mix because both are “the matrix” and both are symmetric. The fix is to write the formula’s matrix explicitly (Σ⁻¹ for the exponent, |Σ| for the determinant) and to check the pdf’s width against the intended covariance by computing the 1σ ellipsoid’s axes √λ_i and comparing.

Self-explanation 6. Before reading on: in the near-miss, if Σ = σ² I (an isotropic covariance with std σ), what is the near-miss’s standard deviation?

Answer. The correct pdf is exp(−½‖x−μ‖²/σ²) / (√(2πσ²))^n, with std σ. The near-miss is exp(−½‖x−μ‖²σ²) / (√(2π/σ²))^n, with std 1/σ. The near-miss’s std is 1/σ — the reciprocal of the correct std. For σ = 0.1, the near-miss’s std is 10 — a hundred-fold error in the variance (0.01 vs. 100). The inverse error is large even when the covariance is small.

§L.3.3 — Comparative table: the four operations on a multivariate Gaussian

Design principle (P9.8). Conditioning shrinks the covariance and marginalizing widens it, and the comparison is PSD, not entrywise. The two operations act on the same matrix and differ only in whether X_b is known or integrated out. The ordering Σ_{aa} − Σ_{ab}Σ_{bb}^{-1}Σ_{ba} ⪯ Σ_{aa} ⪯ Σ_{aa} + Σ_{bb} is what makes “observation reduces uncertainty” a theorem rather than an intuition, and it is the reason a filter’s covariance drops at a measurement and grows between them.

§L.3.4 — Concept check

Concept check (displaces M5). A learner says: “Σ = [[1, 2], [2, 1]] is symmetric, so it is a covariance — I’ll use it in the pdf.” Which requirement has the learner failed to model, and what class of bug will that omission produce?

Answer. The learner has failed to model positive semi-definiteness — the requirement vᵀΣv ≥ 0 for every v, equivalently all eigenvalues ≥ 0. Symmetry is necessary and not sufficient: this matrix has eigenvalues 1 ± 2 = {3, −1}, one of them negative, and det = 1 − 4 = −3. The bug class is a density that is not a density: √|Σ| takes the square root of a negative determinant, so the pdf is NaN or complex, and the exponent (x−μ)ᵀΣ⁻¹(x−μ) is negative along the eigenvector of the negative eigenvalue, so the “density” grows without bound in that direction and integrates to nothing. It is loud at the pdf call if the learner looks at the value, and silent if the value flows into a downstream computation that only checks for exceptions — no exception is raised, and a NaN propagates through every subsequent operation.

§L.3.5 — Consolidation checkpoint 3

(a) You should now be able to: (i) write the multivariate pdf and say why the exponent uses Σ⁻¹ while the normalizer uses |Σ|; (ii) read a covariance’s eigenvalues and eigenvectors as axis lengths and axis directions, and sketch the 1σ ellipsoid; (iii) name the four operations, state which one shrinks the covariance and which one widens it, and identify the Schur complement in the conditional’s formula; (iv) decide whether a given symmetric matrix is admissible as a covariance.

(b) Transfer probe. A learner’s 2D prior is Σ = [[0.04, 0], [0, 0.09]] — uncorrelated — and they condition on q₃ = 0.5. What is the conditional variance of q₂, and how does it compare to WE-L.3.b’s 0.03556? Then they add a third independent error source with covariance [[0.01, 0], [0, 0.01]]: what is the summed covariance? Answer sketch: With Σ_{ab} = 0, the Schur complement’s subtracted term Σ_{ab}Σ_{bb}^{-1}Σ_{ba} = 0, so the conditional variance is the marginal’s 0.04 — the observation is uninformative about q₂, and the mean does not shift either. WE-L.3.b’s correlated case reduced 0.04 to 0.03556; the reduction is exactly the correlation’s contribution, and removing the correlation removes it entirely. The sum is [[0.05, 0], [0, 0.10]] — the variances add blockwise, and the result is PSD because each block is. A learner who reports 0.03556 for the uncorrelated case has applied the formula without noticing that its subtracted term is zero.

(c) Remediation rule. If you cannot do (a)(iii) or (b), return to §L.3.2 and redo WE-L.3.b and WE-L.3.c, then re-attempt (b) by writing the Schur complement out and setting Σ_{ab} = 0 explicitly.

Part D — Bayes’ Rule for Gaussians

§L.4 — Objects: the prior, the likelihood, and the posterior in Gaussian form

For a Gaussian prior p(x) = N(x; μ₀, σ₀²) and a Gaussian likelihood p(y|x) = N(y; x, σ_y²) (the observation is the state plus Gaussian noise), the posterior p(x|y) = N(x; μ₁, σ₁²) with:

The precision form is:

The interpretation:

• The posterior’s precision is the sum of the prior’s and the likelihood’s precisions (the “add” formula).
• The posterior’s mean is the precision-weighted average of the prior’s mean and the observation: the more precise source gets more weight.
• The posterior’s variance is smaller than either the prior’s or the likelihood’s (the update reduces uncertainty).

The multivariate version for x ~ N(μ₀, Σ₀) and y ~ N(x, Σ_y):

The precision matrix Λ = Σ⁻¹ satisfies Λ₁ = Λ₀ + Λ_y (the add formula), and the mean is μ₁ = Λ₁⁻¹ (Λ₀ μ₀ + Λ_y y) — the information-form update.

§L.4.1 — Result: the posterior’s precision is the sum of the prior’s and the likelihood’s; the posterior’s mean is the precision-weighted average; the posterior’s variance is smaller than either

Claim. For p(x) = N(μ₀, σ₀²) and p(y|x) = N(y; x, σ_y²), the posterior is N(μ₁, σ₁²) with 1/σ₁² = 1/σ₀² + 1/σ_y² and μ₁ = σ₁²(μ₀/σ₀² + y/σ_y²). The posterior’s variance σ₁² satisfies σ₁² < min(σ₀², σ_y²), and the posterior’s precision is strictly larger than either precision.

Proof sketch. The posterior’s exponent is the sum of the prior’s and the likelihood’s exponents (the log-posterior is the sum of the log-prior and log-likelihood, up to a constant). Each is a quadratic in x: −(x−μ₀)²/(2σ₀²) and −(y−x)²/(2σ_y²). The sum is a quadratic in x (since the x-dependence is quadratic), and completing the square gives −(x−μ₁)²/(2σ₁²) + const, with 1/σ₁² = 1/σ₀² + 1/σ_y² (the coefficient of x²) and μ₁/σ₁² = μ₀/σ₀² + y/σ_y² (the coefficient of x). The variance’s strict inequality follows from 1/σ₁² > 1/σ₀² and 1/σ₁² > 1/σ_y² (each is the sum of two positive terms, so larger than either), and taking reciprocals reverses the inequality (σ₁² < σ₀² and σ₁² < σ_y²).

Corollary. The posterior’s variance is the harmonic combination of the prior’s and the likelihood’s variances, and it is always smaller than the smaller of the two. The “more data reduces uncertainty” is the sum of precisions — each independent observation adds its precision, and the posterior’s precision grows without bound as the number of observations grows. The 1/√n reduction in std is the square root of the 1/n reduction in variance.

Corollary. The information form Λ₁ = Λ₀ + Λ_y is the basis of the Kalman filter’s information filter (Lesson 0.3.4), and the covariance form Σ₁ = (Σ₀⁻¹ + Σ_y⁻¹)⁻¹ is the basis of the covariance filter. The two are equivalent (one is the matrix inversion of the other), and the choice between them is a computational one (the information form avoids a matrix inversion when Σ_y is easy to invert, and vice versa).

§L.4.2 — Worked-example ladder

WE-L.4.a — Fully worked: the 1D conjugate update, by hand

Input. Prior N(0.5, 0.1²) (mean 0.5, std 0.1), likelihood N(y; x, 0.2²) (std 0.2), measurement y = 0.7.

Trace (precision form).

1. 1/σ₀² = 1/0.01 = 100.
2. 1/σ_y² = 1/0.04 = 25.
3. 1/σ₁² = 100 + 25 = 125 ⇒ σ₁² = 1/125 = 0.008, σ₁ = √0.008 ≈ 0.0894.
4. μ₁/σ₁² = μ₀/σ₀² + y/σ_y² = 0.5·100 + 0.7·25 = 50 + 17.5 = 67.5.
5. μ₁ = 67.5 · σ₁² = 67.5 · 0.008 = 0.54.

Trace (covariance form, for cross-check).

6. σ₁² = σ₀² σ_y² / (σ₀² + σ_y²) = 0.01·0.04 / 0.05 = 0.0004 / 0.05 = 0.008. ✓
7. μ₁ = (σ_y² μ₀ + σ₀² y) / (σ₀² + σ_y²) = (0.04·0.5 + 0.01·0.7) / 0.05 = (0.02 + 0.007) / 0.05 = 0.027 / 0.05 = 0.54. ✓

Output and interpretation. The posterior is N(0.54, 0.008), with std 0.0894 — smaller than both the prior’s 0.1 and the likelihood’s 0.2. The mean 0.54 is closer to the prior’s 0.5 than the measurement’s 0.7, because the prior is more precise (1/σ₀² = 100 vs. 1/σ_y² = 25), so the prior gets 4× the weight. The posterior’s precision 125 is the sum 100 + 25, and the posterior’s variance 0.008 is the harmonic combination.

Self-explanation 7. Before reading on: in step 3, the posterior’s variance is 0.008, which is smaller than the prior’s 0.01. Why is the posterior’s variance smaller than the prior’s, even though the likelihood is less precise than the prior?

Answer. The posterior’s precision is the sum of the prior’s and the likelihood’s, so it is strictly larger than either — even if the likelihood is less precise than the prior, adding its (positive) precision increases the total. The posterior’s variance is the reciprocal of the summed precision, so it is strictly smaller than either individual variance. The intuition “the less precise source should not reduce the variance” is wrong; the correct intuition is “any independent information reduces the uncertainty, and the reduction is proportional to the information’s precision.”

WE-L.4.b — Completion problem

Input. Prior N(0, 1) (std 1), likelihood N(y; x, 4) (std 2), measurement y = 3.

Trace (final 25% as prompts).

1. 1/σ₀² = 1, 1/σ_y² = 1/4 = 0.25. ⟨your turn: verify⟩
2. 1/σ₁² = 1 + 0.25 = 1.25 ⇒ σ₁² = 0.8. ⟨your turn: verify⟩
3. μ₁/σ₁² = 0/1 + 3/4 = 0.75. ⟨your turn⟩
4. μ₁ = 0.75 · 0.8 = 0.6. ⟨your turn⟩

Answers.

1. 1/σ₀² = 1 (variance 1, precision 1); 1/σ_y² = 1/4 = 0.25 (variance 4, precision 0.25).
2. 1.25 precision ⇒ variance 0.8.
3. 0.75 — the numerator’s sum.
4. μ₁ = 0.6 — the posterior’s mean.

The lesson. The posterior’s mean 0.6 is closer to the measurement 3 than the prior’s 0 — but only 20% of the way, because the prior’s precision (1) is 4× the likelihood’s (0.25). The measurement is weak relative to the prior, and the posterior’s mean barely moves. The variance drops from 1 to 0.8 (a 20% reduction), which is the only reduction a single weak measurement can produce.

WE-L.4.c — Faded: the multivariate conjugate update

Setup. The prior on x = [q₂, q₃] is N([0.5, 0.3], Σ₀ = [[0.04, 0.02], [0.02, 0.09]]), and the measurement is y = [0.6, 0.4] with noise covariance Σ_y = [[0.01, 0], [0, 0.01]] (independent, std 0.1 each).

Question. Compute the posterior’s covariance Σ₁ and mean μ₁ by hand.

You must produce the intermediate steps. The setup and the answer are given.

Answer.

• Σ₀⁻¹ = (1/0.0032) [[0.09, -0.02], [-0.02, 0.04]] ≈ [[28.125, -6.25], [-6.25, 12.5]].
• Σ_y⁻¹ = [[100, 0], [0, 100]].
• Λ₁ = Σ₀⁻¹ + Σ_y⁻¹ = [[128.125, -6.25], [-6.25, 112.5]].
• Σ₁ = Λ₁⁻¹ = (1/det) [[112.5, 6.25], [6.25, 128.125]], with det = 128.125·112.5 − 6.25² ≈ 14414 − 39 ≈ 14375. So Σ₁ ≈ [[0.00783, 0.000435], [0.000435, 0.00891]].
• Λ₀ μ₀ = [[28.125, -6.25], [-6.25, 12.5]] · [0.5, 0.3] = [14.0625 − 1.875, -3.125 + 3.75] = [12.1875, 0.625].
• Σ_y⁻¹ y = [100·0.6, 100·0.4] = [60, 40].
• Λ₁ μ₁ = [72.1875, 40.625].
• μ₁ = Σ₁ · [72.1875, 40.625] ≈ [0.00783·72.1875 + 0.000435·40.625, 0.000435·72.1875 + 0.00891·40.625] ≈ [0.582, 0.393].

The lesson. The posterior’s mean ≈ [0.582, 0.393] is close to the measurement [0.6, 0.4] (the measurement is more precise than the prior — 100 precision vs. 28 and 12.5), and the posterior’s variance is dominated by the measurement (0.01’s contribution to the 128 precision). The correlation in the prior (0.02 off-diagonal) makes the posterior’s off-diagonal 0.000435 small but non-zero, and the posterior’s covariance is not diagonal even though the measurement’s is.

WE-L.4.d — Contrastive near-miss: adding variances instead of precisions

Correct solution. The posterior’s precision is 1/σ₀² + 1/σ_y², and the variance is the reciprocal.

Near-miss. The posterior’s variance is σ₀² + σ_y² (the sum of the variances), which is larger than either — the opposite of the correct result.

Why the near-miss produces a plausible but incorrect number. The formula σ₁² = σ₀² + σ_y² is the sum of independent random variables’ variance (WE-L.2.c), not the posterior’s variance after observing one. The learner who confuses “the sum of two independent random variables” with “the posterior of one given the other” uses the wrong closure: the sum’s variance grows (more uncertainty in the total), and the posterior’s variance shrinks (less uncertainty after observing). The error is catastrophic — the posterior’s variance is larger than the prior’s, so the “update” increases the uncertainty, which is the opposite of what observation does. The symptom is “the posterior is less certain than the prior”, and it is loud if the learner checks the direction, but silent if they do not.

The discrepancy, stated. The sum’s variance is σ₀² + σ_y² (larger), and the posterior’s variance is 1/(1/σ₀² + 1/σ_y²) (smaller). The two formulas are reciprocals of each other in a specific sense: the sum is the variance of the sum, and the posterior is the variance of the conditional. The confusion is the variance vs. precision asymmetry (M4 in the inventory), and the fix is to always ask “is this the sum or the condition?” before writing the formula.

Self-explanation 8. Before reading on: if the prior is N(0, 1) and the likelihood is N(y; x, 1), what is the correct posterior’s variance, and what is the near-miss’s “posterior” variance? What is the ratio?

Answer. Correct: 1/(1/1 + 1/1) = 1/2 = 0.5. Near-miss: 1 + 1 = 2. The near-miss’s “posterior” variance is 4× the correct one — the near-miss is less certain than the prior (variance 2 > 1), while the correct posterior is more certain (variance 0.5 < 1). The direction is opposite, and the error is the asymmetry between the sum and the condition.

§L.4.3 — Comparative table: the two algebraic forms of the conjugate update

Design principle (P9.9). Write the update in the form the data arrives in. If observations arrive one at a time, the precision form is a running sum and costs nothing to extend; if a single posterior is wanted, the covariance form avoids an intermediate reciprocal. The two are algebraically identical — WE-L.4.a computes 0.008 and 0.54 both ways — so the choice is about which quantity the surrounding code already holds, not about which is correct.

§L.4.4 — Concept check

Concept check (displaces M3). A learner says: “The posterior is a compromise between the prior and the measurement, so its variance should land somewhere between the two — 0.01 and 0.04 should give something like 0.025.” Which quantity has the learner confused, and what class of bug will that confusion produce?

Answer. The learner has applied the “between” intuition to the variance, when it belongs to the mean. The mean is indeed between μ₀ and y — WE-L.4.a’s 0.54 lies between 0.5 and 0.7 — because it is a precision-weighted average of the two. The variance is the reciprocal of a sum of precisions, 1/(100 + 25) = 0.008, which is smaller than either input: 0.008 < 0.01 < 0.04. The bug class is systematic overconfidence in the wrong direction: a learner who computes σ₁² = 0.025 reports a posterior less certain than the prior, so a filter built on it diverges instead of converging, and a confidence interval built on it is wider than the data warrants. Nothing raises — 0.025 is a positive number in the right units — and the error is only visible against the inequality σ₁² < min(σ₀², σ_y²), which is the one check that catches it.

§L.4.5 — Consolidation checkpoint 4

(a) You should now be able to: (i) derive the posterior’s variance and mean by completing the square in the exponent, and read off 1/σ₁² as the coefficient of x²; (ii) compute the posterior both ways for a concrete 1D problem and confirm the two agree; (iii) state and justify the strict inequality σ₁² < min(σ₀², σ_y²), and explain why a less precise likelihood still reduces the posterior’s variance; (iv) write the multivariate update in precision form and say what each matrix must be.

(b) Transfer probe. Two independent sensors observe the same q₂: sensor A reports y_A = 0.7 with σ_A² = 0.04, sensor B reports y_B = 0.62 with σ_B² = 0.01. The prior is N(0.5, 0.01). Fuse all three, and state the posterior’s std. Then: what would the answer be if a learner fused the two sensors first and then the prior? Answer sketch: Precisions: prior 100, A 25, B 100. Total 225, so σ₁² = 1/225 ≈ 0.00444 and σ₁ ≈ 0.0667 — smaller than every input, including the prior’s 0.1. The weighted numerator is 0.5·100 + 0.7·25 + 0.62·100 = 50 + 17.5 + 62 = 129.5, so μ₁ = 129.5/225 ≈ 0.5756. Fusing the sensors first gives the same answer, because the precision sum is associative: (100 + 25 + 100) does not depend on the order, and the numerator is the same sum of weighted terms. That associativity is exactly what makes the update usable in a filter, where the prior changes every step and the observations arrive in an arbitrary order.

(c) Remediation rule. If you cannot do (a)(iii) or (b), return to §L.4.2 and redo WE-L.4.d, then re-attempt (b) by writing all three precisions before computing anything.

Part E — Failure Taxonomy, Lab, Cheat Sheet, Exercises, Exit Test

§L.5 — Failure taxonomy

§L.6 — The lab

§L.6.1 — Task

Implement gaussian_pdf, mvn_pdf, bayes_update_1d, and bayes_update_mvn; use them to verify the conjugate update, the Schur complement, and the 68–95–99.7 rule; then break the pipeline three ways.

§L.6.2 — Conditions

Python 3.10+, PyTorch 2.2+, CPU. Seed 42, dtype = float64. Runtime under 2 minutes.

§L.6.3 — Controls held fixed

The seed, dtype, the prior N(0.5, 0.1²), the likelihood N(y; x, 0.2²), the measurement y = 0.7, the 2D covariance Σ = [[0.04, 0.02], [0.02, 0.09]], and the q₃ observation 0.5.

§L.6.4 — Metrics table

§L.6.5 — Pre-registered hypotheses

• H1. pdf_0 ≈ 0.3989 and pdf_1 ≈ 0.2420, with the ratio ≈ 0.6065. The prediction is the pair of values.
• H2. rule_1sigma ≈ 0.683 (the 68–95–99.7 rule’s first entry). The prediction is 0.683 ± 0.001.
• H3. post_var = 0.008 and post_mean = 0.54 (from WE-L.4.a). The prediction is the pair of exact values.
• H4. conditional_var = 0.03556 < marginal_var = 0.04 (the conditional is smaller). The prediction is the pair, with the inequality.

§L.6.6 — The break-it sequence

1. The variance-sum bug (silent). Replace 1/σ₁² = 1/σ₀² + 1/σ_y² with σ₁² = σ₀² + σ_y². Predicted symptom: no exception; the “posterior” variance is larger than the prior’s (the opposite of the correct direction). Actual: as predicted — silent.
2. The precision-inversion bug (silent). Use Σ in the exponent instead of Σ⁻¹. Predicted symptom: no exception; the pdf is a different Gaussian with covariance Σ⁻¹, and the 1σ ellipsoid’s axes are inverted. Actual: as predicted — silent.
3. The non-PSD covariance (silent, but produces a NaN). Set Σ = [[0.04, 0.1], [0.1, 0.09]] (det < 0). Predicted symptom: the pdf’s √|Σ| is the square root of a negative number, so the pdf is NaN. Actual: as predicted — silent (no exception at the pdf call, but the value is NaN).

Two of the three are silent (the variance-sum and the precision-inversion); the third produces a NaN, which is loud only if the learner checks.

§L.6.7 — Acceptance criteria

§L.6.8 — What good looks like

A strong submission has five properties:

1. The pdf values are computed and printed with the standard normal’s constants (1/√(2π) and exp(−0.5)), so the mechanism is visible, not just the output.
2. The 68–95–99.7 rule is verified by integration, not assumed, and the integration’s value matches the rule.
3. The conjugate update is computed in both the precision and covariance forms, and the two agree.
4. The conditional’s variance is shown to be smaller than the marginal’s, and the difference is attributed to the Schur complement.
5. The three break-its are predicted before running, with the direction of the bug (e.g., “larger variance” vs. “smaller”) stated explicitly.

§L.6.9 — Runnable lab code

# lab_0_3_1.py — Bayes' rule, Gaussians, and the conjugate update
# Requires: Python 3.10+, PyTorch 2.2+

import torch, math
torch.manual_seed(42); dtype, device = torch.float64, torch.device("cpu")

# --- Step 1: the univariate Gaussian pdf ------------------------------
def gaussian_pdf(x, mu, var):
    """Univariate Gaussian pdf at x."""
    # torch.sqrt on a Python float raises TypeError; keep the scale a tensor.
    scale = 1.0 / torch.sqrt(torch.tensor(2 * math.pi * var, dtype=dtype, device=device))
    return scale * torch.exp(-((x - mu) ** 2) / (2 * var))

print(f"p(0) for N(0,1): {gaussian_pdf(torch.tensor(0., dtype=dtype), 0., 1.).item():.4f}")
print(f"p(1) for N(0,1): {gaussian_pdf(torch.tensor(1., dtype=dtype), 0., 1.).item():.4f}")

# --- Step 2: the 68-95-99.7 rule by integration ----------------------
def integrate_pdf(mu, var, a, b, n=10000):
    xs = torch.linspace(a, b, n, dtype=dtype)
    dx = (b - a) / (n - 1)
    return (gaussian_pdf(xs, mu, var) * dx).sum().item()

print(f"P(|Z|<1): {integrate_pdf(0., 1., -1., 1.):.4f}")
print(f"P(|Z|<2): {integrate_pdf(0., 1., -2., 2.):.4f}")
print(f"P(|Z|<3): {integrate_pdf(0., 1., -3., 3.):.4f}")

# --- Step 3: the 1D conjugate update ----------------------------------
def bayes_update_1d(mu0, var0, y, var_y):
    """Precision form of the Gaussian conjugate update."""
    prec0, prec_y = 1.0 / var0, 1.0 / var_y
    prec1 = prec0 + prec_y
    var1 = 1.0 / prec1
    mu1 = var1 * (mu0 * prec0 + y * prec_y)
    return mu1, var1

mu0, var0 = 0.5, 0.01
y, var_y = 0.7, 0.04
mu1, var1 = bayes_update_1d(mu0, var0, y, var_y)
print(f"posterior: mean = {mu1:.6f}, var = {var1:.6f}, std = {math.sqrt(var1):.6f}")

# Covariance form, for cross-check
var1_cov = var0 * var_y / (var0 + var_y)
mu1_cov = (var_y * mu0 + var0 * y) / (var0 + var_y)
print(f"covariance-form check: mean = {mu1_cov:.6f}, var = {var1_cov:.6f}")

# --- Step 4: the multivariate Gaussian pdf ----------------------------
def mvn_pdf(x, mu, Sigma):
    """Multivariate Gaussian pdf at x (n,)."""
    d = x - mu
    inv = torch.linalg.inv(Sigma)
    det = torch.linalg.det(Sigma)
    n = x.shape[0]
    return (1.0 / torch.sqrt((2 * math.pi) ** n * det)) * torch.exp(-0.5 * d @ inv @ d)

Sigma = torch.tensor([[0.04, 0.02], [0.02, 0.09]], dtype=dtype)
mu = torch.tensor([0.5, 0.3], dtype=dtype)
x = torch.tensor([0.5, 0.3], dtype=dtype)
print(f"mvn_pdf at mean: {mvn_pdf(x, mu, Sigma).item():.6f}")

# --- Step 5: eigenvalues and eigenvectors ----------------------------
eigvals, eigvecs = torch.linalg.eigh(Sigma)
print(f"eigenvalues: {eigvals.tolist()}, eigenvectors:\n{eigvecs}")
print(f"1-sigma semi-axes: {torch.sqrt(eigvals).tolist()}")

# --- Step 6: the 2D conditional (Schur complement) --------------------
Sigma_aa = Sigma[0, 0]; Sigma_ab = Sigma[0:1, 1:2]; Sigma_bb = Sigma[1, 1]
cond_var = (Sigma_aa - Sigma_ab @ torch.linalg.inv(Sigma_bb.reshape(1, 1)) @ Sigma_ab.T).item()
cond_mean = (mu[0] + (Sigma_ab @ torch.linalg.inv(Sigma_bb.reshape(1, 1)) * (0.5 - mu[1]).reshape(1, 1)).squeeze()).item()
print(f"marginal var of q2: {Sigma_aa.item():.6f}")
print(f"conditional var of q2 | q3=0.5: {cond_var:.6f}")
print(f"conditional mean of q2 | q3=0.5: {cond_mean:.6f}")

# --- Step 7: break-it #1 — variances added instead of precisions -----
def bayes_update_wrong(mu0, var0, y, var_y):
    var1_wrong = var0 + var_y  # WRONG: should be 1/(1/var0 + 1/var_y)
    mu1_wrong = (var0 * mu0 + var_y * y) / (var0 + var_y)  # also wrong: simple average
    return mu1_wrong, var1_wrong

mu_w, var_w = bayes_update_wrong(mu0, var0, y, var_y)
print(f"[break-it #1] wrong posterior: mean = {mu_w:.6f}, var = {var_w:.6f}")
print(f"[break-it #1] the wrong var ({var_w}) is {'LARGER' if var_w > var0 else 'smaller'} than the prior's ({var0})")

# --- Step 8: break-it #2 — Sigma vs. Sigma^{-1} in the exponent ----
def mvn_pdf_wrong(x, mu, Sigma):
    d = x - mu
    det_wrong = torch.linalg.det(torch.linalg.inv(Sigma))  # WRONG: det of Sigma^{-1}
    n = x.shape[0]
    return (1.0 / torch.sqrt((2 * math.pi) ** n * det_wrong)) * torch.exp(-0.5 * d @ Sigma @ d)

x_test = torch.tensor([0.7, 0.5], dtype=dtype)
print(f"[break-it #2] correct pdf: {mvn_pdf(x_test, mu, Sigma).item():.6f}")
print(f"[break-it #2] wrong pdf: {mvn_pdf_wrong(x_test, mu, Sigma).item():.6f}")

# --- Step 9: break-it #3 — non-PSD covariance -----------------------
Sigma_bad = torch.tensor([[0.04, 0.1], [0.1, 0.09]], dtype=dtype)
print(f"[break-it #3] det of bad Sigma: {torch.linalg.det(Sigma_bad).item():.6f} (should be < 0)")
try:
    val = mvn_pdf(x_test, mu, Sigma_bad)
    print(f"[break-it #3] pdf at bad Sigma: {val.item()}")
except Exception as e:
    print(f"[break-it #3] pdf raised: {e}")

# --- Step 10: summary ------------------------------------------------
print("\n--- lab 0.3.1 summary ---")
print(f"H1: p(0)={gaussian_pdf(torch.tensor(0., dtype=dtype), 0., 1.).item():.4f}, p(1)={gaussian_pdf(torch.tensor(1., dtype=dtype), 0., 1.).item():.4f}")
print(f"H2: P(|Z|<1)={integrate_pdf(0., 1., -1., 1.):.4f}")
print(f"H3: posterior mean={mu1:.4f}, var={var1:.4f}")
print(f"H4: conditional var={cond_var:.4f} < marginal var={Sigma_aa.item():.4f}")
print(f"silent break-its: #1 (variance sum), #2 (Sigma vs Sigma^-1); #3 is NaN")

§L.7 — The cheat sheet

§L.8 — Exercises with solutions

E 0.3.1.1 — Bayes’ rule (numerical). Prior p(q=0)=0.3, p(q=1)=0.7; likelihood p(y|q=0)=0.9, p(y|q=1)=0.1. Compute the posterior.

S 0.3.1.1. Unnormalized: [0.27, 0.07], sum 0.34. Posterior: [0.794, 0.206]. The lesson: the prior’s small advantage for q=1 is overwhelmed by the likelihood’s strong evidence for q=0, and the posterior flips to favor q=0.

E 0.3.1.2 — The 68–95–99.7 rule (numerical). For N(2, 0.5²), what is P(X ∈ [1.5, 2.5])? [1, 3]? [0.5, 3.5]?

S 0.3.1.2. Standardize: [−1, 1], [−2, 2], [−3, 3]. So the probabilities are 0.683, 0.954, 0.997. The lesson: the rule is invariant to the mean and variance after standardization; the actual X interval changes with the parameters, but the Z interval is fixed.

E 0.3.1.3 — The conjugate update (numerical). Prior N(1, 1), likelihood N(y; x, 0.25) (var 0.25), measurement y = 2. Compute the posterior.

S 0.3.1.3. Precision form: 1/σ₀² = 1, 1/σ_y² = 4, 1/σ₁² = 5, σ₁² = 0.2. μ₁/σ₁² = 1·1 + 2·4 = 9, so μ₁ = 9·0.2 = 1.8. The posterior is N(1.8, 0.2). The lesson: the posterior’s mean 1.8 is closer to the measurement 2 than the prior’s 1, because the likelihood’s precision (4) is 4× the prior’s (1). The posterior’s variance 0.2 is smaller than both (1 and 0.25).

E 0.3.1.4 — The Schur complement (numerical). Σ = [[1, 0.5], [0.5, 1]], conditioning on X_b = 0. Compute the conditional’s variance of X_a.

S 0.3.1.4. Σ_{aa} = 1, Σ_{ab} = 0.5, Σ_{bb} = 1. Schur complement: 1 − 0.5·1·0.5 = 1 − 0.25 = 0.75. The conditional’s variance is 0.75, smaller than the marginal’s 1 — the observation of X_b reduces X_a’s uncertainty by 0.25. The lesson: the conditional’s variance is smaller than the marginal’s, and the reduction is the correlation squared (0.5² = 0.25 for Σ_{bb} = 1).

E 0.3.1.5 — Density vs. probability (diagnostic). A learner reports “the probability that q = 0.5 is 3.99” for a posterior N(0.5, 0.1²). What is the error, and what is the correct probability?

S 0.3.1.5. The error is the density vs. probability confusion. p(0.5 | y) = 3.99 is the density at the point (per unit q), not the probability that q = 0.5, which is zero (a point has zero probability under a continuous distribution). The correct statement is p(q ∈ [0.45, 0.55] | y) ≈ 0.38, computed by integrating the pdf over the interval. The lesson: the density is a rate (probability per unit q), and the probability is an integral (density × width); the density can exceed 1, and the probability cannot.

E 0.3.1.6 — Variance vs. precision (diagnostic). A learner’s posterior’s variance is 2, and the prior’s is 1. What is the most likely bug, and what is the correct variance if the likelihood’s variance is 1?

S 0.3.1.6. The bug is adding variances instead of precisions: the learner computed σ₁² = 1 + 1 = 2. The correct posterior’s variance is 1/(1/1 + 1/1) = 0.5 — smaller than the prior’s 1, not larger. The lesson: the posterior’s variance is smaller than either input’s; a larger posterior’s variance is the sum formula, not the condition formula, and the bug is the asymmetry.

E 0.3.1.7 — The non-PSD covariance (diagnostic). A learner’s mvn_pdf returns NaN for Σ = [[1, 2], [2, 1]]. What is the cause, and what is the fix?

S 0.3.1.7. det(Σ) = 1·1 − 2·2 = 1 − 4 = -3 < 0, so Σ is not PSD, and √|Σ| is the square root of a negative number → NaN. The eigenvalues are 1 ± 2 = {3, -1} — one negative — which is the cause. The fix is to use a PSD covariance (e.g., Σ = [[1, 0.5], [0.5, 1]], whose det = 0.75 > 0), or to project the matrix onto the PSD cone (eigen-decompose, clip the eigenvalues to ≥ 0, and re-synthesize). The lesson: a covariance must be PSD, and the check is det > 0 and all eigenvalues ≥ 0; a non-PSD matrix is not a covariance, and the pdf’s NaN is the loud symptom of the silent bug.

E 0.3.1.8 — The multivariate update (constructive, transfer probe). Write the pseudocode for a kalman_update function that takes (μ₀, Σ₀, y, Σ_y) and returns (μ₁, Σ₁). State what the function’s Σ_y must satisfy.

S 0.3.1.8.

function kalman_update(mu_0, Sigma_0, y, Sigma_y):
    Lambda_0 = inv(Sigma_0)
    Lambda_y = inv(Sigma_y)
    Lambda_1 = Lambda_0 + Lambda_y
    Sigma_1 = inv(Lambda_1)
    mu_1 = Sigma_1 @ (Lambda_0 @ mu_0 + Lambda_y @ y)
    return mu_1, Sigma_1

The Σ_y must be symmetric positive-definite (a covariance), and the same for Σ₀. The function is the information-form update, and it is the batch version of the Kalman filter’s measurement update (Lesson 0.3.4). The lesson: the conjugate update is a matrix expression that reduces to the 1D formula, and its inputs are covariances (PSD), its output is a covariance (PSD), and its correctness is the precision-sum rule Λ₁ = Λ₀ + Λ_y applied to matrices.

§L.9 — Exit test

You are ready for Lesson 0.3.2 when you can, without notes:

1. State the four terms of Bayes’ rule and their roles, for discrete and continuous random variables.
2. State the proportional form and explain when it is sufficient and when it is not.
3. Write the univariate Gaussian’s pdf, and compute its value at a specific point by hand.
4. State the 68–95–99.7 rule and verify it for a specific Gaussian.
5. State the three properties of the univariate Gaussian (affine closure, sum closure, standardization) and apply each.
6. Write the multivariate Gaussian’s pdf, and interpret the covariance’s eigenvalues and eigenvectors.
7. State the four multivariate operations (linear map, conditioning, marginalization, sum) and their formulas.
8. Derive the 1D Gaussian conjugate update in precision form, and compute the posterior’s mean and variance for a specific problem.
9. State the multivariate conjugate update in precision form (Λ₁ = Λ₀ + Λ_y).
10. State the “precisions add; variances do not” asymmetry, and diagnose the “posterior variance larger than the prior” bug.
11. State the “density vs. probability” distinction, and give a symptom of the confusion.
12. State the “PSD covariance” requirement and the symptom of a non-PSD covariance (the NaN from √|Σ|).

If any item is not answerable without notes, return to the section that introduced it: 1 to §L.1, 2 to §L.1, 3 to §L.2, 4 to §L.2, 5 to §L.2.1, 6 to §L.3, 7 to §L.3.1, 8 to §L.4, 9 to §L.4.1, 10 to §L.4.4 and E 0.3.1.6, 11 to §L.2.4 and E 0.3.1.5, 12 to §L.3.2 and E 0.3.1.7.

§L.10 — Provenance and evidence tiers

What this lesson reproduces. Every hypothesis in §L.6.5 is executed by the lab in §L.6.9, and the numbers in the text are the ones the lab prints. The lab is the evidence, so these claims are tier A, status R.

What this lesson does not reproduce. The claim that the 68–95–99.7 rule’s remaining entries are 0.9545 and 0.9973 is stated in the text and not printed by the lab, which computes only the first. It is tier B, status S — a standard result, stated without a run. The same holds for the closed form μ_post = (σ₀²y + σ²μ₀)/(σ₀² + σ²); the lab checks one instance of it (H3), not the identity.

The break-it taxonomy is the lesson’s main result, and it is a measurement. Two of the three break-its are silent and one is loud. The distinction is not a stylistic choice: the silent pair (Σ for Σ⁻¹, variance summed instead of inverted) produce finite, plausible numbers that pass any check written to look for exceptions, while the non-PSD Σ fails at the square root. A reader who tests only for exceptions will find the third and miss the first two.

No external sources are cited in this lesson. Every result is either a definition, an integral the reader can check by hand, or a behavior the lab measures. The Gaussian identities are standard and are reproduced rather than cited; the lesson names no paper, and this section does not invent one.