X0.3.3

Lesson 0.3.3 — Filtering vs. Smoothing: The Causal and Acausal Estimators, and Why the Kalman Filter Is Not a Smoother

Week 4 · Prerequisites: 0.3.1, 0.3.2, 0.2.1, 0.1.2.

Prior results used. This lesson uses 0.3.1’s Bayes’ rule and Gaussian conjugate update, 0.3.2’s ML/MAP distinction and the log-posterior, 0.2.1’s SE(3) composition (for the pose-tracking example), and 0.1.2’s autograd (for the gradient check on the smoother’s loss). Each is restated at the point of use.

Hands forward. This lesson hands forward one capability: the learner can state the filtering, prediction, and smoothing problems, write the general Bayes-filter recursion, derive the Kalman filter’s predict and update equations, derive the RTS smoother’s backward pass, distinguish causal from acausal estimates, and diagnose the four canonical filter/smoother bugs (missing Q, missing smoother cross-covariance, filter/smoother confusion, wrong gain) from their symptoms — the capability that Lesson 0.3.4 (the Kalman filter’s implementation) and every state-estimation, SLAM, and control lesson in Phase 1–4 assumes.

Running case. The same 6-DOF MuJoCo arm from 0.2.1–0.3.2, now tracked over time: the state is x_t = [q₂(t), q̇₂(t)]ᵀ (joint-2’s angle and angular velocity), the dynamics are constant-velocity with process noise, the observation is the encoder’s noisy angle, and the three inference problems are (i) filtering p(x_t | y_{1:t}) (estimate the current state given all past measurements), (ii) prediction p(x_{t+k} | y_{1:t}) (estimate a future state given past measurements), and (iii) smoothing p(x_t | y_{1:T}) (estimate a past state given all measurements, including future ones). The lesson’s central result is that (iii) is more accurate than (i) for offline tasks, less causal than (i) for online tasks, and the Kalman filter is (i), not (iii).

§L.0 — Lesson map, objectives, and misconceptions

§L.0.1 — Lesson map

§L.0.2 — Objectives

By the end of this lesson you will be able to do eight things:

1. State the three inference problems (filtering, prediction, smoothing) and the conditioning set each uses, and classify a given robot-estimation task into one of the three.
2. Write the general Bayes filter’s predict and update equations, and state the Markov and conditional-independence assumptions each relies on.
3. Derive the Kalman filter’s predict (μ⁻ = Aμ, Σ⁻ = AΣAᵀ + Q) and update (K = Σ⁻Hᵀ(HΣ⁻Hᵀ + R)⁻¹, μ = μ⁻ + K(y − Hμ⁻), Σ = (I − KH)Σ⁻) equations, and compute one predict-update step by hand.
4. State the Kalman gain’s interpretation as the relative precision of the prior and the measurement, and compute its limit as R → 0 and R → ∞.
5. Write the RTS smoother’s backward recursion (C_t = Σ_t Aᵀ (Σ_{t+1}⁻)⁻¹, μ_t^s = μ_t + C_t(μ_{t+1}^s − μ_{t+1}⁻), Σ_t^s = Σ_t + C_t(Σ_{t+1}^s − Σ_{t+1}⁻)C_tᵀ), and explain why the smoothed covariance is smaller than the filtered covariance.
6. Distinguish causal (filter) from acausal (smoother) estimates, and state the online vs. offline use cases.
7. Diagnose four filter/smoother bugs — missing Q, missing smoother cross-covariance, filter/smoother confusion, wrong gain — from their symptoms.
8. Produce a written “estimator audit” of a robot pipeline: the state, the dynamics, the observation model, the inference problem, the estimator, and the online/offline constraint.

§L.0.3 — Misconception inventory

Each misconception is displaced in the section named.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

This is the eleventh lesson. The prompt requires drawing from lessons 2–5 steps back; 0.3.1 and 0.3.2 are 1–2 steps back and are excluded. The warm-up draws from 0.2.4 (four steps back), 0.2.3 (five steps back), and 0.1.2 (nine steps back, but a prerequisite of 0.3.2).

Warm-up 1 (from 0.2.4, the frame conventions). In a multi-frame pipeline, what is the difference between T_A^B and T_B^A, and what is the composition’s rule? Answer: T_A^B maps B-expressed quantities to A-expressed; T_B^A = (T_A^B)^{-1} is the reverse; the composition is T_A^C = T_A^B · T_B^C, with the intermediate frame canceling. How this is used: §L.3’s Kalman filter’s state can be a pose, and the predict step’s A is the pose’s transformation under the dynamics — the same T_A^B-style composition, with the state’s frame being the “time” axis and the dynamics being the “motion.”

Warm-up 2 (from 0.2.3, the adjoint and the twist). A twist ξ = (ω, v) is a 6-vector; the state’s velocity is a twist. What is a twist’s transformation law, and why is it not a 4×4 multiply? Answer: The twist transforms by the adjoint Ad_T (a 6×6 matrix), ξ_A = Ad_{T_A^B} ξ_B, with a cross-product term in the linear part; the 4×4 T does not transform a twist (it transforms a point or a direction, not a velocity). How this is used: §L.3’s state includes the velocity q̇₂, and the Kalman filter’s A matrix is the linearized version of the twist’s propagation — the adjoint’s 6×6 form, linearized around the current estimate.

Warm-up 3 (from 0.1.2, autograd and the graph). A loss is a sum of log-posteriors over timesteps. Does autograd’s chain rule flow through the sum? Answer: Yes — the sum is a chain of differentiable operations, and .backward() accumulates the gradient from each term. How this is used: §L.4’s smoother is the maximum of the joint log-posterior over the whole trajectory, and its gradient (through the whole chain) is the batch optimization that the RTS smoother’s closed form solves analytically for the linear-Gaussian case.

Warm-up 4 (from 0.2.4, the “effect in the code” pattern). A codebase writes p_W = T_WB @ p_B. What does the name T_WB mean in the from-to convention, and what is the inverse’s name? Answer: T_WB is the from-B-to-W transform; the inverse is T_BW. How this is used: §L.3’s Kalman filter’s matrices A, H, Q, R are frame-agnostic (they are matrices on the state’s vector space), but the state’s frame and the measurement’s frame must match the model, and the mismatch is the source of the “wrong H” bug (which is not in the four canonical bugs but is the frame audit’s concern).

Part A — The Three Inference Problems

§L.1 — Objects: filtering, prediction, smoothing, and their conditioning sets

Given a hidden state sequence x_{1:T} = (x_1, …, x_T) and an observation sequence y_{1:T} = (y_1, …, y_T), there are three inference problems:

1. Filtering — estimate the current state given past and present measurements:

It is causal (uses only y_1, …, y_t) and is the estimator for online tasks (control, tracking, SLAM’s front-end).

2. Prediction — estimate a future state given past measurements:

It is causal and is the estimator for planning (a controller predicting the state at t + Δ), lookahead (a policy evaluating future states), and anomaly detection (a predicted state’s deviation from the observed state).

3. Smoothing — estimate a past state given all measurements:

It is acausal (uses y_{t+1}, …, y_T) and is the estimator for offline tasks (map-building, post-hoc trajectory analysis, batch SLAM’s back-end, training-data labeling).

The conditioning sets distinguish the three:

The key insight: the smoother uses more data than the filter (it adds the future measurements), so the smoothed estimate is more accurate — its covariance is smaller. But the smoother is not available at time t (the future measurements do not exist yet), so it cannot be used online. The choice between filtering and smoothing is the causality constraint, not the accuracy.

§L.1.1 — Result: the smoother’s covariance is PSD-smaller than the filter’s, and the difference is the information from future measurements

Claim. For a linear-Gaussian state-space model, the smoothed covariance Σ_t^s at time t satisfies Σ_t^s ⪯ Σ_t (PSD-smaller), where Σ_t is the filtered covariance. The difference Σ_t − Σ_t^s is the information contributed by the future measurements y_{t+1:T}, and it is non-negative (PSD).

Proof sketch. The smoothing problem conditions on y_{1:T}, which is a superset of the filtering problem’s y_{1:t}. Conditioning on additional variables reduces (or leaves unchanged) the covariance — this is the same Schur-complement result as 0.3.1’s conditioning operation: Σ_{x|y_{1:T}} ⪯ Σ_{x|y_{1:t}} because the conditioning set is larger. The RTS smoother’s backward pass computes this conditioning explicitly, and the smoothed covariance is the filtered covariance minus the C_t(Σ_{t+1}^s − Σ_{t+1}⁻)C_tᵀ term, which is PSD (since Σ_{t+1}^s ⪯ Σ_{t+1}⁻ by the same induction).

Corollary. The smoother is more accurate at every timestep, but the accuracy gain is largest for the middle of the trajectory (where both past and future contribute) and zero at the last timestep (t = T, where the smoother has no future data and equals the filter). The smoothing gain grows with the number of future measurements, and it decays as the measurements’ influence decays (the Markov chain “forgets” the past).

Corollary. The filter’s estimate at time t and the smoother’s estimate at time t are different estimates of the same state, and the smoother’s is not a “correction” of the filter’s for a bug — it is a different (better) estimator for the offline problem. The filter is not wrong; it is causal, which is a constraint, not a bug.

§L.1.2 — Worked-example ladder

WE-L.1.a — Fully worked: the three problems for the joint-2 trajectory

Input. A joint-2 trajectory of T = 5 timesteps, with measurements y = [0.5, 0.7, 0.6, 0.8, 0.4] (the running case’s readings from 0.3.2). The true states are unknown, but assume (for the example) the true angles are q₂ = [0.5, 0.65, 0.6, 0.7, 0.5].

Trace (filtering at each t).

1. t = 1: p(x₁ | y₁) uses y₁ = 0.5.
2. t = 2: p(x₂ | y₁, y₂) uses y₁ = 0.5, y₂ = 0.7.
3. t = 3: p(x₃ | y₁, y₂, y₃) uses all three.
4. t = 4: p(x₄ | y₁, …, y₄) uses all four.
5. t = 5: p(x₅ | y₁, …, y₅) uses all five.

Trace (prediction).

6. From t = 1, predict x₂: p(x₂ | y₁) uses only y₁ = 0.5.
7. From t = 1, predict x₅: p(x₅ | y₁) uses only y₁ = 0.5, with four steps of dynamics-only propagation.

Trace (smoothing).

8. For every t = 1, …, 5, p(x_t | y₁, …, y₅) uses all five measurements, including the ones after t.

Output and interpretation. The filter at t = 3 uses y_{1:3}; the predictor at t = 3 from t = 1 uses y₁; the smoother at t = 3 uses y_{1:5}. The smoother’s estimate of q₂(3) is influenced by y₄ = 0.8 and y₅ = 0.4, which is future information — physically, the smoother “knows” that the trajectory ends at 0.4, which is lower than the filtered estimate 0.6, and the smoothed estimate is pulled toward the lower end. The pull is the future information’s effect, and it is not available to the filter at t = 3.

Self-explanation 1. Before reading on: in the smoothing case, p(x₃ | y_{1:5}) uses y₄ and y₅, which are future relative to t = 3. Is the smoother “cheating”? Why or why not?

Answer. The smoother is not cheating — it is solving a different problem. The filter is for online use (at time t, only y_{1:t} exists); the smoother is for offline use (at the end of the trajectory, all y_{1:T} exist, and the question is “what was x₃?”). The smoother’s use of y_{4:5} is legitimate for the offline problem, and it produces a more accurate estimate of x₃ than the filter’s — but that estimate is not available at time 3, so it cannot be used for online control at time 3. The smoother is “cheating” only if the learner uses it for an online task, which is the filter/smoother confusion (M2).

WE-L.1.b — Completion problem

Input. The same trajectory, but the learner wants to estimate x₄ at time t = 2.

Trace (final 25% as prompts).

1. Is this filtering, prediction, or smoothing? ⟨your turn⟩
2. Which measurements does the estimate use? ⟨your turn⟩
3. What is the estimate’s covariance compared to the smoother’s estimate of x₄ (from t = 2 to T = 5)? ⟨your turn⟩

Answers.

1. Prediction — the target x₄ is future relative to t = 2, and the estimate uses y_{1:2}.
2. y₁ and y₂ only.
3. The prediction’s covariance is larger than the smoother’s, because the prediction uses fewer measurements (y_{1:2} vs. y_{1:5}) and the dynamics’ process noise Q adds uncertainty at every step. The prediction’s covariance grows with the prediction horizon k, and the smoother’s is bounded.

The lesson. Prediction is a special case of filtering (with the target at t + k and the data at t), and its covariance grows with the horizon. Smoothing is the acausal case, and its covariance shrinks with more future data. The three problems have different covariance behaviors, and the choice between them is the causality constraint plus the horizon.

WE-L.1.c — Faded: the running case with the prediction horizon

Setup. The filter at t = 1 has estimate μ₁ = 0.5 and covariance σ₁² = 0.01 (a scalar, for simplicity). The dynamics are q₂(t+1) = q₂(t) + q̇₂(t)·dt with q̇₂ = 0 (constant position, for this example) and process noise Q = 0.001 per step. The dynamics matrix A = 1 (the identity in 1D).

Question. What is the predicted covariance at t = 2, t = 3, t = 5, t = 10?

You must produce the intermediate steps. The setup and the answer are given.

Answer. Σ₂⁻ = AΣ₁Aᵀ + Q = 0.01 + 0.001 = 0.011; Σ₃⁻ = 0.011 + 0.001 = 0.012; Σ₅⁻ = 0.01 + 4·0.001 = 0.014; Σ₁₀⁻ = 0.01 + 9·0.001 = 0.019. The covariance grows linearly with the horizon (in the 1D case), reaching 0.019 at t = 10 — nearly double the starting 0.01.

The lesson. The prediction’s covariance grows with the horizon k as Σ + k·Q (for a 1D identity dynamics), and it grows without bound as k → ∞ (the process noise accumulates). The prediction is useful only for short horizons, and its covariance’s growth is the predictability limit — a controller that predicts too far ahead is overconfident and wrong.

WE-L.1.d — Contrastive near-miss: using the filter for an offline task

Correct solution. A learner building a map from a recorded trajectory uses the smoother p(x_t | y_{1:T}) for every t, because the map-building is offline (all data is available) and the smoother is more accurate.

Near-miss. The learner uses the filter p(x_t | y_{1:t}) for the same offline task, because “the filter is the standard Kalman filter.”

Why the near-miss produces a plausible but incorrect number. The filter’s estimates are legal (each p(x_t | y_{1:t}) is a valid distribution), and the map built from them is mostly correct — the filter’s estimates are close to the smoother’s for most timesteps, and the map’s shape is approximately right. But the accuracy is lower (the filter’s covariance is larger), and the errors are systematic (the filter’s trajectory lags the true one, and the lag is visible in the map’s features). The error is silent (the map looks fine), and the cost is a map that is less accurate than it could be, with the lag distorting the geometry.

The discrepancy, stated. The filter’s estimate at time t uses y_{1:t}, and the smoother’s uses y_{1:T}. The difference is the future information, which is non-zero whenever the dynamics couple x_t and x_{t+1} (which they always do) and the future measurements are informative about x_t (which they always are, through the dynamics). The filter’s estimate is the best causal estimate; the smoother’s is the best estimate, and for an offline task, the “best” is the smoother’s.

Self-explanation 2. Before reading on: in the near-miss, if the dynamics’ process noise Q = 0 (a perfectly deterministic system), is the filter still worse than the smoother?

Answer. Yes — even with Q = 0, the filter’s estimate at t uses y_{1:t}, and the smoother’s uses y_{1:T}. The future measurements y_{t+1:T} are functions of the state x_t (through the dynamics), so they carry information about x_t, and the smoother uses that information. The gain is larger with Q > 0 (because the process noise makes the future measurements more informative about the past, since the past’s uncertainty is larger), but it is non-zero even with Q = 0. The only case with zero gain is when the future measurements are uninformative about x_t (e.g., H = 0 for the future), which is a degenerate measurement model, not the typical case.

§L.1.3 — Comparative table: the three inference problems

Design principle (P11.6). Budget the prediction horizon against Q, not against the model’s smoothness. The predicted covariance grows by the process noise once per step, so a horizon of k steps costs k·Q of variance before the model’s fidelity is even in question. A controller that predicts ten steps ahead with Q = 0.001 carries k·Q = 0.01 of added variance at the horizon, and the loss is computable before the run: the horizon is a design parameter with a price, and the price is k·Q per predicted quantity. What that addition costs relatively depends on the covariance you started from — 0.01 doubles a Σ₀ = 0.01 and is a rounding error against a Σ₀ = 1.0 — so the number to design against is the ratio k·Q / Σ₀, and stating the horizon without it is stating half a budget. Choosing the horizon by how far ahead the task “needs” to look, without pricing it, is how a lookahead controller acquires a confident estimate of a state it cannot see.

§L.1.4 — Concept check

Concept check (displaces M2). A learner says: “The smoother is just a better filter — the future measurements can only help — so I’ll run it on the live stream and use its estimates instead of the filter’s.” What has the learner failed to model, and what does the resulting bug look like?

Answer. The learner has failed to model the conditioning set as a constraint on availability: the smoother at time t conditions on y_{1:T}, and the measurements after t do not exist at t. Running it “on the live stream” means one of two things, and both are defects. Either the estimate is delayed by T − t steps — a control loop acting on it acts on a state the arm has already left — or the backward pass is re-run every step over a growing trajectory, so the estimate of x_t changes retroactively while the controller has already acted on the previous value. The bug class is acausal feedback: every number the loop consumes is a valid estimate of something, and none of them is an estimate of the state the actuator is in. Offline the numbers look excellent, which is why the defect survives review — the covariance is smaller, the tracking error is smaller, and only the loop’s timing or its closed-loop stability reveals the mismatch.

§L.1.5 — Consolidation checkpoint 1

(a) You should now be able to: (i) name the three inference problems by their conditioning sets, and state which direction of time each uses; (ii) classify a given robot task into one of the three from its online/offline constraint rather than from its accuracy requirement; (iii) state the covariance behavior of each — growth with the horizon, reduction with future data, equality at t = T; (iv) state why the smoother’s smaller covariance is a different problem’s answer and not a correction of the filter’s.

(b) Transfer probe. A warehouse robot runs a filter live for ten minutes. After the run, an engineer replays the logged data through the RTS smoother and obtains a trajectory with uniformly smaller covariance. A colleague concludes: “the filter was buggy; the smoother is the fix — ship the smoother.” Answer sketch: the filter was not buggy; it answered p(x_t | y_{1:t}) and the smoother answered p(x_t | y_{1:T}), and the smaller covariance is the future information, not a repair. The filter is the right estimator for the live loop and the wrong one for the post-hoc analysis; the smoother is the reverse. Shipping the smoother means either adding T − t steps of latency or letting the estimate of a past state drive a present command. The correct reading of the replay is a diagnostic, not a patch: the gap between the two covariances is the value of the future data, and it tells you how much the live loop is paying for causality.

(c) Remediation rule. If you cannot do (b), return to §L.1.2 and redo WE-L.1.d, then re-attempt (b) by asking which measurements exist at the instant the controller acts.

Part B — The General Bayes Filter

§L.2 — Objects: the Markov assumption and the two-step recursion

The Bayes filter is the recursive computation of the filtering distribution p(x_t | y_{1:t}) from the previous filtering distribution p(x_{t-1} | y_{1:t-1}), the dynamics p(x_t | x_{t-1}), and the observation p(y_t | x_t).

The recursion has two steps:

1. Predict — propagate the previous filtering distribution through the dynamics:

This is the Chapman–Kolmogorov equation, and it is the marginalization of x_{t-1}.

2. Update — incorporate the new measurement y_t via Bayes’ rule:

The denominator is a normalizer, and the posterior is proportional to the product of the likelihood p(y_t | x_t) and the predicted prior p(x_t | y_{1:t-1}).

The assumptions the recursion relies on:

• Markov (state completeness): p(x_t | x_{t-1}, y_{1:t-1}) = p(x_t | x_{t-1}) — the state summarizes the past, so the dynamics depend only on the previous state. This is the state-completeness assumption, and it is violated when the state is not a complete summary (e.g., a state that omits a slowly-varying parameter).
• Conditional independence of measurements: p(y_t | x_t, y_{1:t-1}) = p(y_t | x_t) — the measurement depends only on the current state, not on the past. This is the measurement-locality assumption, and it is violated when the sensor’s noise is correlated across time (which is common: a biased sensor’s bias is a slowly-varying state, not white noise).

The Bayes filter is a recursion, not a batch algorithm — it computes p(x_t | y_{1:t}) incrementally, using only the previous distribution and the new measurement, and it is the causal algorithm for the filtering problem.

§L.2.1 — Result: the predict step’s uncertainty grows (the dynamics add noise), and the update step’s uncertainty shrinks (the measurement adds information); the filter’s covariance is the balance

Claim. In the Bayes filter, the predict step’s uncertainty Σ⁻ is PSD-larger than the previous step’s Σ (the dynamics add noise and stretch the covariance), and the update step’s uncertainty Σ is PSD-smaller than Σ⁻ (the measurement adds information). The filter’s covariance converges to a steady-state where the predict’s growth equals the update’s reduction, and the steady-state covariance is strictly positive (not zero).

Proof sketch. The predict step is Σ⁻ = AΣAᵀ + Q, with Q ⪰ 0; the AΣAᵀ is a linear map’s transformation, and the +Q adds the process noise’s uncertainty, so Σ⁻ ⪰ AΣAᵀ ⪰ 0. The update step is the conditioning on y_t, which is the Schur-complement reduction (from 0.3.1), giving Σ ⪯ Σ⁻. The steady state is the fixed point of the composed map Σ_{k+1} = f(Σ_k), and its existence and uniqueness are the standard Kalman filter’s convergence result (under detectability and stabilizability). The strict positivity of the steady-state covariance is because Q ≻ 0 (the process noise is always adding uncertainty) and the measurement H is not perfectly informative (the R ≻ 0).

Corollary. The “more measurements reduce uncertainty” is not unconditional — the update reduces it, but the predict increases it, and the net is a balance. A filter that omits Q (M4) has no growth in the predict, so the covariance shrinks to zero and the filter stops trusting new measurements — the classic overconfidence bug.

Corollary. The steady-state covariance Σ_∞ and the steady-state gain K_∞ are constants, and the filter converges to a linear time-invariant estimator. The steady-state gain is the optimal fixed gain for the steady-state covariance, and it is what a steady-state Kalman filter uses (the “alpha-beta filter” and its variants are the steady-state KF for specific dynamics).

§L.2.2 — Worked-example ladder

WE-L.2.a — Fully worked: one Bayes filter recursion for the joint-2 model, by hand

Input. State x_t = q₂(t) (a scalar for this example, ignoring velocity), dynamics x_{t+1} = x_t + w_t with w_t ~ N(0, Q = 0.01), observation y_t = x_t + v_t with v_t ~ N(0, R = 0.04). Previous filtering distribution N(μ = 0.5, σ² = 0.01). New measurement y_t = 0.7.

Trace (predict).

1. μ⁻ = Aμ = 1·0.5 = 0.5 (the dynamics do not move the mean; only the noise is added).
2. Σ⁻ = AΣAᵀ + Q = 1·0.01·1 + 0.01 = 0.02.

Trace (update).

3. S = HΣ⁻Hᵀ + R = 0.02 + 0.04 = 0.06.
4. K = Σ⁻Hᵀ / S = 0.02 / 0.06 ≈ 0.3333.
5. μ = μ⁻ + K(y − Hμ⁻) = 0.5 + 0.3333·(0.7 − 0.5) = 0.5 + 0.0667 = 0.5667.
6. Σ = (1 − K·H)Σ⁻ = (1 − 0.3333)·0.02 ≈ 0.0133.

Output and interpretation. The predict’s covariance 0.02 is larger than the previous 0.01 (the process noise added 0.01). The update’s covariance 0.0133 is smaller than the predicted 0.02 (the measurement reduced the uncertainty by 0.0067). The update’s mean 0.5667 is between the prior 0.5 and the measurement 0.7, with the gain 0.3333 — the measurement is twice as precise as the prior (R = 0.04 vs. Σ⁻ = 0.02, so the prior is more precise, and the gain should be < 0.5).

The gain is higher when the measurement is more precise. Σ⁻ = 0.02, R = 0.04 — the prior’s precision is 1/0.02 = 50, the measurement’s is 1/0.04 = 25 — so the prior is more precise, and the gain is less than 0.5. K = 0.02/0.06 ≈ 0.333, which is < 0.5. ✓

Self-explanation 3. Before reading on: in step 4, K = 0.333 means the update moves the mean 33% of the way toward the measurement. What is the gain when R → 0 (a perfect measurement), and when R → ∞ (a useless measurement)?

Answer. As R → 0, S → Σ⁻, and K = Σ⁻/Σ⁻ = 1 — the filter fully trusts the measurement and the estimate becomes the measurement. As R → ∞, S → ∞, and K → 0 — the filter ignores the measurement and the estimate stays at the prior. The gain is the relative precision of the prior and the measurement, interpolating between the two extremes.

WE-L.2.b — Completion problem

Input. Same model, but with Q = 0.001 (smaller process noise) and R = 0.01 (more precise measurement). Previous N(0.5, 0.01), new y_t = 0.7.

Trace (final 25% as prompts).

1. Σ⁻ = AΣAᵀ + Q = 0.01 + 0.001 = 0.011. ⟨your turn: verify⟩
2. S = Σ⁻ + R = 0.011 + 0.01 = 0.021. ⟨your turn⟩
3. K = Σ⁻/S = 0.011 / 0.021. ⟨your turn: compute⟩
4. μ = 0.5 + K(0.7 − 0.5). ⟨your turn⟩
5. Σ = (1 − K)·Σ⁻. ⟨your turn⟩

Answers.

1. 0.011. ✓
2. 0.021. ✓
3. K ≈ 0.5238.
4. μ ≈ 0.5 + 0.5238·0.2 = 0.6048.
5. Σ ≈ (1 − 0.5238)·0.011 ≈ 0.4762·0.011 ≈ 0.00524.

The lesson. With a smaller Q (0.001 vs. 0.01) and a more precise R (0.01 vs. 0.04), the gain is higher (0.5238 vs. 0.3333) — the filter trusts the measurement more — and the updated covariance is smaller (0.00524 vs. 0.0133) — the estimate is more certain. The gain’s value is the relative precision, and its changes track the changes in Q and R.

WE-L.2.c — Faded: the steady-state gain

Setup. Same model as WE-L.2.a (Q = 0.01, R = 0.04, A = 1, H = 1), run for many steps until convergence.

Question. What is the steady-state covariance Σ_∞ and the steady-state gain K_∞?

You must produce the intermediate steps. The setup and the answer are given.

Answer. The steady state satisfies Σ_∞ = (1 − K_∞)·Σ_∞⁻ and Σ_∞⁻ = Σ_∞ + Q and K_∞ = Σ_∞⁻ / (Σ_∞⁻ + R). Substituting: Σ_∞ = (1 − Σ_∞⁻/(Σ_∞⁻+R))·Σ_∞⁻ = (R/(Σ_∞⁻+R))·Σ_∞⁻ = R·Σ_∞⁻/(Σ_∞⁻+R). And Σ_∞⁻ = Σ_∞ + Q. Two equations, two unknowns. Solving: Σ_∞ = R(Σ_∞ + Q)/(Σ_∞ + Q + R) ⇒ Σ_∞(Σ_∞ + Q + R) = R(Σ_∞ + Q) ⇒ Σ_∞² + Σ_∞(Q + R) = RΣ_∞ + RQ ⇒ Σ_∞² + Σ_∞·Q − RQ = 0 ⇒ Σ_∞ = (-Q + √(Q² + 4RQ))/2. With Q = 0.01, R = 0.04: Σ_∞ = (-0.01 + √(0.0001 + 0.0016))/2 = (-0.01 + √0.0017)/2 = (-0.01 + 0.0412)/2 ≈ 0.0156. K_∞ = Σ_∞⁻/(Σ_∞⁻ + R) = (Σ_∞ + Q)/(Σ_∞ + Q + R) = 0.0256/(0.0256 + 0.04) = 0.0256/0.0656 ≈ 0.390.

The lesson. The steady-state covariance 0.0156 is between the process noise’s Q = 0.01 and the measurement noise’s R = 0.04, and the steady-state gain 0.390 is less than 0.5 (the prior is more precise). The asymptotic behavior is a constant covariance and gain, which is the steady-state filter. The transient behavior (the first few steps) depends on the initial covariance, but the steady state does not.

WE-L.2.d — Contrastive near-miss: missing Q

Correct solution. The predict step is Σ⁻ = AΣAᵀ + Q.

Near-miss. The predict step is Σ⁻ = AΣAᵀ (omitting Q).

Why the near-miss produces a plausible but incorrect number. The near-miss’s predict step shrinks the covariance (or leaves it unchanged), and the update step shrinks it further — the covariance monotonically decreases to zero. As the covariance approaches zero, the gain K = Σ⁻/(Σ⁻ + R) → 0, so the filter stops trusting new measurements and ignores them. The filter’s estimate freezes at its initial value (or at the mean of the first few measurements), and the true state drifts away while the filter’s estimate stays stuck. The error is silent at first (the filter looks “confident”), and it is catastrophic as the state drifts — the filter diverges from the truth, and the divergence is invisible until the state is far away.

The discrepancy, stated. Σ⁻ = AΣAᵀ + Q vs. Σ⁻ = AΣAᵀ differ by Q, the process noise’s covariance. The Q is the only source of new uncertainty in the predict step, and its omission makes the covariance a strictly decreasing sequence that converges to zero — which is physically wrong (a real system’s uncertainty never goes to zero, because the process noise is always present). The Q is the filter’s acknowledgment that the state changes, and its omission is the overconfidence bug (M4).

Self-explanation 4. Before reading on: in the near-miss, if Q = 0 genuinely (the dynamics are deterministic), does the filter still diverge?

Answer. If Q = 0 genuinely — the dynamics x_{t+1} = Ax_t are exactly satisfied, with no process noise — then the covariance’s convergence to zero is correct, and the filter is optimal. But real robotic systems never have exactly zero process noise: the model is an approximation (unmodeled dynamics, parameter uncertainty, external disturbances), and the discrepancy between the model and the reality is the effective Q. A learner who sets Q = 0 because “the model is exact” is wrong — the model is never exact, and the Q is the residual uncertainty that the model does not capture. The Q’s value should be tuned to the residual uncertainty, not the true process noise (which is unknowable).

§L.2.3 — Comparative table: the Bayes filter’s assumptions and their violations

Design principle (P11.7). Test the assumptions on the innovations, not on the model. The Markov and locality assumptions are statements about the residuals, and the residuals are the only place the violations are observable: a state that is not complete leaves a structured innovation sequence, and correlated sensor noise leaves an autocorrelated one, while the filter’s internal covariance looks healthy in both cases. The audit is mechanical — plot the innovation, its autocorrelation, and its normalized squared value against the χ²(m) band — and it costs one line in the logging code. A filter validated only by “the estimate looks right” has tested the model against itself.

§L.2.4 — Concept check

Concept check (displaces M5). A learner says: “The update’s precision-sum Λ_post = Λ_prior + Λ_meas shows the precision only ever grows, so more measurements mean less uncertainty — the covariance falls monotonically.” Where does the argument break, and what does the resulting bug look like?

Answer. The argument is correct about the update and silent about the predict, which happens between the updates: the predict maps Σ through A and adds Q, and both operations can increase the covariance. The filter’s covariance is therefore the fixed point of a two-step map, not a decreasing sequence, and it converges to a strictly positive steady state — 0.0156 for the lesson’s scalar model with Q = 0.01 and R = 0.04, between the two noise levels. The learner who carries the monotone intuition into the design sets Q = 0 (or omits it), because the predict “should not grow anything”, and the covariance then does fall to zero, the gain follows it, and the estimate freezes while the state drifts. The bug class is self-inflicted overconfidence: the filter becomes certain of a stale value, and the certainty is invisible because the covariance is doing exactly what the learner’s model predicted.

§L.2.5 — Consolidation checkpoint 2

(a) You should now be able to: (i) write the predict and update equations and name the assumption each relies on; (ii) state the direction of the covariance’s change at each step — the predict grows it, the update shrinks it — and why the steady state is strictly positive; (iii) compute one scalar recursion by hand, including the gain and both covariances; (iv) state why Q is the residual uncertainty rather than the true process noise, and what the omission produces.

(b) Transfer probe. A learner tunes a filter on a bench rig with a rigidly mounted encoder and finds Q = 0 gives the lowest tracking error, so they ship Q = 0 to a production cell where the same encoder is mounted on a flexing bracket. Answer sketch: on the bench the model is nearly exact, so the residual uncertainty is genuinely small and a small Q is optimal; on the flexing bracket the mounting introduces dynamics the model does not contain, and the effective Q is the variance of that unmodeled motion, which is not small. With Q = 0 the shipped filter’s covariance collapses, the gain follows, and the estimate freezes while the bracket flexes — the bench result is not wrong, it is an answer about a different plant. The correct move is to set Q from the residual on the production data (the innovation sequence’s own variance is the estimate) rather than to carry the bench value across, and to re-tune whenever the mechanical mounting changes.

(c) Remediation rule. If you cannot do (b), return to §L.2.2 and redo WE-L.2.d, then re-attempt (b) by asking which motions exist on the bracket that the model’s A does not contain.

Part C — The Kalman Filter

§L.3 — Objects: the linear-Gaussian model and the closed-form predict-update

The linear-Gaussian state-space model is:

with A the dynamics matrix, B the control matrix, u_t the control input, Q the process noise covariance, H the observation matrix, and R the measurement noise covariance. All matrices are known (or estimated), and the noises are independent across time (white).

The Kalman filter is the Bayes filter for this model, and it is closed-form because the linear-Gaussian model preserves Gaussianity:

Predict:

Update:

The innovation y − Hμ⁻ is the difference between the measurement and the predicted measurement; the gain K is the weight on the innovation; the update’s mean is the predicted mean plus the weighted innovation. The covariance is the predicted covariance minus the KH term, which is the information the measurement added.

The gain’s interpretation:

• K → I·H^{-1} (or K → H^{-1} in the square case) when R → 0 — the measurement is perfect, and the estimate becomes the measurement.
• K → 0 when R → ∞ or Σ⁻ → 0 — the measurement is useless or the prior is certain, and the estimate stays at the prior.
• K = Σ⁻Hᵀ(HΣ⁻Hᵀ + R)^{-1} is the relative precision of the prior and the measurement, in matrix form.

§L.3.1 — Result: the Kalman filter is the exact Bayes filter for linear-Gaussian models, and the optimal (minimum mean-square) estimator among all linear estimators for any noise (with the Gaussian giving the exact MMSE among all estimators)

Claim. For the linear-Gaussian model, the Kalman filter’s μ is the exact conditional mean E[x_t | y_{1:t}] and the Σ is the exact conditional covariance Cov[x_t | y_{1:t}]. For non-Gaussian noise with the same mean and covariance, the KF is the best linear estimator (the minimum mean-square error among linear estimators), but not necessarily the best overall.

Proof sketch. The joint distribution p(x_t, y_{1:t}) is Gaussian (a linear map of Gaussian noises), and the conditional p(x_t | y_{1:t}) is Gaussian with mean and covariance given by the standard Gaussian conditioning formula (from 0.3.1). The Kalman filter’s recursion is the recursive form of that conditioning, and its μ and Σ are the conditional mean and covariance. The linear-optimality for non-Gaussian noise is the orthogonality principle (the linear estimator’s error is orthogonal to the measurements), and the KF satisfies it by construction.

Corollary. The KF’s optimality is conditional on the model (linear dynamics, linear observation, known A, H, Q, R). A wrong Q or R makes the KF suboptimal (and possibly divergent), even though the recursion is exact for the assumed model. The “wrong Q” bug (M4) is the model’s error, not the recursion’s.

Corollary. For nonlinear dynamics or observation, the KF’s linearity is a limitation: the EKF linearizes around the current estimate (the Jacobian F = ∂f/∂x replaces A), the UKF propagates sigma points through the nonlinearity, and the particle filter samples from the full posterior. All three approximate the Bayes filter, and their optimality is not guaranteed in the same sense as the KF’s.

§L.3.2 — Worked-example ladder

WE-L.3.a — Fully worked: one KF predict-update cycle, by hand

Input. State x = [q₂, q̇₂]ᵀ, A = [[1, 0.1], [0, 1]] (constant velocity with dt = 0.1), B = 0, Q = [[0.001, 0], [0, 0.01]] (small position noise, larger velocity noise), H = [1, 0] (observe position only), R = 0.04. Previous estimate μ = [0.5, 0.1]ᵀ, Σ = [[0.01, 0], [0, 0.04]]. Measurement y_t = 0.7.

Trace (predict).

1. μ⁻ = Aμ = [[1, 0.1], [0, 1]]·[0.5, 0.1] = [0.5 + 0.01, 0.1] = [0.51, 0.1].
2. AΣAᵀ = [[1, 0.1], [0, 1]]·[[0.01, 0], [0, 0.04]]·[[1, 0], [0.1, 1]]. First AΣ = [[0.01, 0.004], [0, 0.04]], then (AΣ)Aᵀ = [[0.01 + 0.004·0.1, 0.004], [0.04·0.1, 0.04]] = [[0.0104, 0.004], [0.004, 0.04]].
3. Σ⁻ = [[0.0104, 0.004], [0.004, 0.04]] + [[0.001, 0], [0, 0.01]] = [[0.0114, 0.004], [0.004, 0.05]].

Trace (update).

4. S = HΣ⁻Hᵀ + R = 0.0114 + 0.04 = 0.0514.
5. K = Σ⁻Hᵀ/S = [0.0114, 0.004]ᵀ / 0.0514 = [0.2218, 0.0778]ᵀ.
6. μ = μ⁻ + K(y − Hμ⁻) = [0.51, 0.1] + [0.2218, 0.0778]·(0.7 − 0.51) = [0.51 + 0.0421, 0.1 + 0.0148] = [0.5521, 0.1148].
7. Σ = (I − KH)Σ⁻. KH = [0.2218, 0.0778]ᵀ·[1, 0] = [[0.2218, 0], [0.0778, 0]]. I − KH = [[0.7782, 0], [-0.0778, 1]]. (I − KH)Σ⁻ = [[0.7782·0.0114, 0.7782·0.004], [-0.0778·0.0114 + 0.004, -0.0778·0.004 + 0.05]] = [[0.00887, 0.00311], [0.00311, 0.04969]].

Output and interpretation. The predict step moved the mean (the velocity 0.1 shifted the position by 0.01) and grew the covariance (the process noise added). The update step shifted the mean toward the measurement (0.5521 vs. the predicted 0.51) and shrunk the covariance. The position covariance dropped from 0.0114 to 0.00887, and the velocity covariance from 0.05 to 0.0497 — the measurement mostly informs the position, and the velocity is informed indirectly (through the correlation 0.00311).

Self-explanation 5. Before reading on: in step 5, the gain’s velocity component is 0.0778, which is non-zero even though the observation is only of position (H = [1, 0]). Why?

Answer. The velocity is correlated with the position (the covariance’s off-diagonal Σ⁻[0, 1] = 0.004 is non-zero, because the dynamics couple the two: the position depends on the velocity). The measurement of the position is informative about the velocity through the correlation, and the gain’s velocity component is the correlation scaled by the precision. When the correlation is zero (e.g., a diagonal Σ⁻), the gain’s velocity component is zero, and the position measurement does not inform the velocity. The correlation is the source of the cross-information, and it is the 0.3.1 conditioning’s Σ_{ab}Σ_{bb}^{-1} term in the KF’s form.

WE-L.3.b — Completion problem

Input. Same model, but with Σ = [[0.04, 0], [0, 0.04]] (larger initial uncertainty, no correlation). Measurement y = 0.7.

Trace (final 25% as prompts).

1. μ⁻ = Aμ = [0.5, 0.1] + [0.01, 0] = [0.51, 0.1]. ⟨your turn⟩
2. AΣAᵀ = [[0.04, 0.004], [0.004, 0.04]]. ⟨your turn: verify⟩
3. Σ⁻ = [[0.041, 0.004], [0.004, 0.05]]. ⟨your turn⟩
4. S = 0.041 + 0.04 = 0.081. ⟨your turn⟩
5. K = [0.041, 0.004]ᵀ/0.081 ≈ [0.506, 0.049]ᵀ. ⟨your turn⟩
6. μ = [0.51, 0.1] + [0.506, 0.049]·(0.7 − 0.51) = [0.51 + 0.096, 0.1 + 0.0093] = [0.606, 0.109]. ⟨your turn⟩

Answers.

1. [0.51, 0.1]. ✓
2. AΣAᵀ = [[1·0.04 + 0.1·0, 1·0 + 0.1·0.04], [0·0.04 + 1·0, 0·0 + 1·0.04]] = [[0.04, 0.004], [0.004, 0.04]]. ✓
3. Σ⁻ = [[0.04 + 0.001, 0.004], [0.004, 0.04 + 0.01]] = [[0.041, 0.004], [0.004, 0.05]]. ✓
4. S = 0.041 + 0.04 = 0.081. ✓
5. K ≈ [0.506, 0.049]. ✓
6. μ ≈ [0.606, 0.109]. ✓

The lesson. With a larger initial uncertainty (0.04 vs. 0.01), the gain is higher (0.506 vs. 0.222) — the filter trusts the measurement more because the prior is less certain — and the updated mean moves more toward the measurement (0.606 vs. 0.552). The gain’s dependency on the prior’s covariance is the KF’s adaptive behavior, and it is what the steady-state approximation loses (the steady-state gain is fixed, and it is suboptimal if the true covariance differs from the steady-state).

WE-L.3.c — Faded: the KF for the running case’s full 6-DOF pose

Setup. The running case’s full state is x = [pose (6D), velocity (6D)]ᵀ (12D), and the observation is the camera’s 6D pose of the end-effector. The matrices A, H, Q, R are the block-diagonal extensions of the 2D case.

Question. What is the dimension of Σ, and what is the dimension of K? What is the cost of one KF step?

You must produce the intermediate steps. The setup and the answer are given.

Answer. Σ is 12 × 12 (state covariance), K is 12 × 6 (state × observation). The KF step’s dominant cost is the S inversion, which is 6 × 6 (observation covariance) — O(6³) = O(216) operations, negligible for a modern CPU. The AΣAᵀ is 12 × 12 × 12 = O(12³) = O(1728), also negligible. The KF’s cost is cubic in the state and observation dimensions, which is why the EKF/UKF’s linearization (Jacobians) is the only extra cost, and the particle filter’s cost is linear in the number of particles but much larger for high-dimensional states.

The lesson. The KF’s cost is cubic in the dimensions, so it is real-time for states up to ~100D (the 100³ = 10^6 operations per step is < 1 ms on a modern CPU). For higher-dimensional states (e.g., a dense SLAM’s 10^4 landmarks), the sparsity of the matrices (block-diagonal, banded) is exploited, or the information filter (which updates the precision matrix, sparse for SLAM) is used. The choice between the covariance and information forms is the sparsity of the problem.

WE-L.3.d — Contrastive near-miss: the gain with R instead of S

Correct solution. K = Σ⁻Hᵀ(HΣ⁻Hᵀ + R)^{-1} — the denominator is S, the innovation’s covariance.

Near-miss. K = Σ⁻Hᵀ R^{-1} — the denominator is R, the measurement’s covariance, ignoring the prior’s contribution.

Why the near-miss produces a plausible but incorrect number. The near-miss’s K is larger than the correct one (since R ⪯ S), so the filter over-trusts the measurement and under-weights the prior. The estimate jumps to the measurement more than it should, and the covariance shrinks too fast — the filter becomes overconfident and loses the prior’s regularization. For a single measurement with a flat prior (Σ⁻ → ∞), the two are close (the prior is negligible), but for a informative prior (small Σ⁻), the near-miss is wrong by the ratio (Σ⁻ + R)/R, which is large when Σ⁻ ≫ R — the opposite of the case where the two are close.

The discrepancy, stated. K = Σ⁻Hᵀ/(HΣ⁻Hᵀ + R) vs. K = Σ⁻Hᵀ/R. The correct denominator is S = HΣ⁻Hᵀ + R, the total uncertainty in the innovation (the prior’s and the measurement’s); the near-miss’s denominator is R alone, the measurement’s uncertainty, and it ignores the prior’s. The near-miss is correct only when Σ⁻ → 0 (the prior is certain), which is the opposite of the case where the gain matters. The near-miss is the “ignoring the prior in the gain” bug, and it is common in code that adapts a scalar filter’s formula (K = σ²/(σ² + R)) to a matrix without tracking the HΣ⁻Hᵀ term.

Self-explanation 6. Before reading on: in the near-miss, if Σ⁻ = R (the prior and the measurement are equally uncertain), what is the correct K and the near-miss’s K?

Answer. Correct: K = Σ⁻/(Σ⁻ + R) = Σ⁻/(2Σ⁻) = 0.5 (the half weight). Near-miss: K = Σ⁻/R = 1 (the full weight). The near-miss doubles the gain, and the filter fully trusts the measurement even though the prior is equally informative. The error is the HΣ⁻Hᵀ term’s omission, and it is systematic (the gain is always too high), not random.

§L.3.3 — Comparative table: the gain’s two candidate denominators and the gain’s limits

Design principle (P11.8). Write the gain as a ratio of precisions and check the ratio’s direction before the code runs. The gain is Σ⁻HᵀS⁻¹, and S = HΣ⁻Hᵀ + R is the total uncertainty in the innovation — the prior’s contribution plus the measurement’s. Any implementation that divides by R alone has silently asserted that the prior contributes nothing, and the assertion is testable in one line: perturb Σ⁻ upward and check that K rises. A gain that does not respond to the prior’s covariance is the R-denominator bug wearing a passing test’s clothes, and the response check costs less than the debugging it replaces.

§L.3.4 — Concept check

Concept check (displaces M1). A learner says: “The Kalman filter is the optimal state estimator, so for my arm’s nonlinear end-effector observation I’ll use it directly — the matrices just need to be the arm’s kinematics.” What has the learner failed to model, and what does the resulting bug look like?

Answer. The learner has failed to model where the optimality comes from: the KF’s μ and Σ are the exact conditional mean and covariance of a linear-Gaussian model, and the exactness is a property of the Gaussian conditioning, not of the recursion’s shape. Substituting a nonlinear h(q) for H produces a filter that is neither exact nor optimal: the innovation y − h(μ⁻) is no longer Gaussian, the gain is no longer the conditional mean’s slope, and the covariance is no longer the conditional covariance. The bug class is silent suboptimality that reads as tuning: the filter runs, converges, and produces plausible numbers, and the residual error is attributed to Q and R — which the learner then tunes against a model error, driving the noise parameters away from their physical values to compensate for a linearization that was never valid. The fix is to linearize (EKF, with H = ∂h/∂q at the current estimate) or to propagate sigma points (UKF), and the diagnosis is that no choice of Q and R makes a nonlinear observation linear.

§L.3.5 — Consolidation checkpoint 3

(a) You should now be able to: (i) write the KF’s predict and update and identify each matrix’s role; (ii) compute one 2D predict-update by hand, including the gain’s cross-component structure; (iii) state the gain’s interpretation as relative precision, and both limits (R → 0 and Σ⁻ → 0); (iv) state the condition under which the KF is exact, and what changes when the model is nonlinear.

(b) Transfer probe. A learner’s 2D filter observes position only (H = [1, 0]) and finds the velocity estimate never improves, no matter how many measurements arrive; the gain’s velocity component prints as exactly zero. The covariance starts diagonal. Answer sketch: the velocity is informed only through the position-velocity correlation, since H selects the position alone; with a diagonal Σ⁻ the cross term is zero, so Σ⁻Hᵀ has a zero second component and K’s velocity entry is zero. The filter then never creates the correlation from the measurement, and the loop is closed: no correlation, no velocity information; no velocity information, no correlation. The lesson’s own model does not have this problem because its A = [[1, dt], [0, 1]] couples the two components in the predict, and AΣAᵀ puts a non-zero (0, 1) entry there — 0.004 in WE-L.3.a — which is what lets the position measurement reach the velocity. A learner who reads the zero velocity gain as “the measurement says nothing about velocity” has the mechanism backwards: the measurement says nothing about velocity because the predict failed to couple them.

(c) Remediation rule. If you cannot do (b), return to §L.3.2 and redo WE-L.3.a, then re-attempt (b) by asking where the off-diagonal entry of Σ⁻ comes from.

Part D — The RTS Smoother

§L.4 — Objects: the forward-backward recursion and the cross-covariance

The Rauch–Tung–Striebel (RTS) smoother computes the smoothed distributions p(x_t | y_{1:T}) in two passes:

1. Forward pass: run the Kalman filter, storing μ_t, Σ_t, μ_t⁻, Σ_t⁻ for every t = 1, …, T.
2. Backward pass: for t = T − 1, …, 1, compute:

with the initial condition μ_T^s = μ_T and Σ_T^s = Σ_T (the smoother equals the filter at the last timestep).

The cross-covariance C_t is the smoother’s gain: it is the sensitivity of x_t to x_{t+1}, and it is the 0.3.1 conditioning’s Σ_{ab}Σ_{bb}^{-1} term for the state’s temporal chain. The backward pass propagates the future information backward, and the smoothed estimate at t is the filtered estimate plus a correction that depends on the difference between the smoothed and predicted estimates at t + 1.

§L.4.1 — Result: the smoother’s backward pass is the exact computation of p(x_t | y_{1:T}) for linear-Gaussian models, and its covariance is smaller than the filter’s

Claim. For the linear-Gaussian model, the RTS smoother’s μ_t^s and Σ_t^s are the exact E[x_t | y_{1:T}] and Cov[x_t | y_{1:T}]. The smoothed covariance Σ_t^s ⪯ Σ_t (PSD-smaller than the filtered), with equality at t = T and strict inequality for t < T when the future measurements are informative.

Proof sketch. The joint distribution p(x_t, x_{t+1} | y_{1:t}) is Gaussian (a linear map of the noise), and the backward conditional p(x_t | x_{t+1}, y_{1:T}) (which is the smoother’s intermediate) is Gaussian with mean μ_t + C_t(x_{t+1} − μ_{t+1}⁻) and covariance Σ_t − C_t Σ_{t+1}⁻ C_tᵀ, by the standard Gaussian conditioning. Substituting the smoothed x_{t+1} distribution p(x_{t+1} | y_{1:T}) and marginalizing gives the RTS recursion, and the covariance is Σ_t + C_t(Σ_{t+1}^s − Σ_{t+1}⁻)C_tᵀ, which is PSD-smaller than Σ_t because Σ_{t+1}^s ⪯ Σ_{t+1}⁻.

Corollary. The smoother’s gain C_t is the cross-covariance between x_t and x_{t+1}, and it is the 0.3.1 Schur-complement’s Σ_{ab}Σ_{bb}^{-1} in the temporal chain. The smoother’s backward pass is the exact version of the “look at the future” that the filter cannot do, and its covariance reduction is the quantitative benefit of using the future.

Corollary. The smoother’s computational cost is twice the filter’s (two passes), and its memory is O(T) (it stores the forward pass’s μ_t, Σ_t, μ_t⁻, Σ_t⁻ for every t). The cost is acceptable for offline tasks (the trajectory’s length is the data’s length, not the runtime’s), but it is not acceptable for online tasks (the future data does not exist).

§L.4.2 — Worked-example ladder

WE-L.4.a — Fully worked: one RTS backward step, by hand

Input. A 2-step trajectory with the 2D model from WE-L.3.a (state [q₂, q̇₂]ᵀ). Filtered at t = 1: μ₁ = [0.5, 0.1]ᵀ, Σ₁ = [[0.01, 0], [0, 0.04]]. Predicted at t = 2: μ₂⁻ = [0.51, 0.1]ᵀ, Σ₂⁻ = [[0.0114, 0.004], [0.004, 0.05]]. Filtered at t = 2: μ₂ = [0.5521, 0.1148]ᵀ, Σ₂ = [[0.00887, 0.00311], [0.00311, 0.04969]].

Trace (the smoother’s cross-covariance).

1. A = [[1, 0.1], [0, 1]].
2. Σ₁ Aᵀ = [[0.01, 0], [0, 0.04]]·[[1, 0], [0.1, 1]] = [[0.01, 0], [0.004, 0.04]].
3. (Σ₂⁻)^{-1} = [[0.0114, 0.004], [0.004, 0.05]]^{-1} = (1/det)·[[0.05, -0.004], [-0.004, 0.0114]] with det = 0.0114·0.05 − 0.004² = 0.00057 − 0.000016 = 0.000554. So (Σ₂⁻)^{-1} ≈ [[90.3, -7.2], [-7.2, 20.6]].
4. C₁ = Σ₁Aᵀ·(Σ₂⁻)^{-1} ≈ [[0.01, 0], [0.004, 0.04]]·[[90.3, -7.2], [-7.2, 20.6]] = [[0.903, -0.072], [0.361 - 0.288, -0.029 + 0.824]] ≈ [[0.903, -0.072], [0.073, 0.795]].

Trace (the smoothed estimate).

5. μ₁^s = μ₁ + C₁(μ₂^s − μ₂⁻) = [0.5, 0.1] + [[0.903, -0.072], [0.073, 0.795]]·([0.5521, 0.1148] − [0.51, 0.1]) = [0.5, 0.1] + [[0.903, -0.072], [0.073, 0.795]]·[0.0421, 0.0148].
6. The correction: [0.903·0.0421 − 0.072·0.0148, 0.073·0.0421 + 0.795·0.0148] ≈ [0.0380 − 0.0011, 0.0031 + 0.0118] ≈ [0.0369, 0.0149].
7. μ₁^s ≈ [0.537, 0.115].

Trace (the smoothed covariance).

8. Σ₁^s = Σ₁ + C₁(Σ₂^s − Σ₂⁻)C₁ᵀ. With Σ₂^s = Σ₂ (at the last timestep, the smoother equals the filter): Σ₂^s − Σ₂⁻ = [[0.00887 − 0.0114, 0.00311 − 0.004], [0.00311 − 0.004, 0.04969 − 0.05]] = [[-0.00253, -0.00089], [-0.00089, -0.00031]].
9. C₁·(Σ₂^s − Σ₂⁻)·C₁ᵀ = [[-0.00195, -0.00078], [-0.00078, -0.00031]], so Σ₁^s = [[0.01, 0], [0, 0.04]] + [[-0.00195, -0.00078], [-0.00078, -0.00031]] = [[0.00805, -0.00078], [-0.00078, 0.03969]].

Output and interpretation. The smoothed mean at t = 1 is [0.537, 0.115], shifted from the filtered [0.5, 0.1] toward the future measurement’s effect. The shift’s direction is the cross-covariance’s direction (the position and velocity together, since they are correlated), and the magnitude is the innovation μ₂^s − μ₂⁻ times the cross-covariance. The smoothed covariance’s position variance falls from 0.0100 to 0.00805 and its velocity variance from 0.0400 to 0.03969 — a 19.5% reduction on the position against 0.8% on the velocity, which is the same asymmetry the filter showed: the position measurement informs the position directly and the velocity only through the correlation. The off-diagonal turns negative (−0.00078 against the filtered 0), because the smoother has seen a future position that is slightly above the predicted one and now reads the velocity as correspondingly lower.

Self-explanation 7. Before reading on: in step 7, the smoothed mean [0.537, 0.115] is between the filtered [0.5, 0.1] and the future [0.552, 0.115]. Is the smoothed estimate “ahead” of the filtered, and why?

Answer. Yes — the smoothed estimate is pulled toward the future, and it is ahead of the filter’s estimate (in the direction of the future state’s evolution). The future data y₂ says the state moved to 0.552, and the smoother back-propagates that information: if the state was at 0.552 at t = 2, and the dynamics couple x₁ and x₂, then x₁ was likely closer to 0.552 than the filter thought. The pull is the cross-covariance’s effect, and it is the temporal version of 0.3.1’s conditioning: the future is a “measurement” of the past, through the dynamics.

WE-L.4.b — Completion problem

Input. Same trajectory, t = 1 smoothing with a larger Q (Q = 0.1 for both position and velocity), so the dynamics are loosely coupled.

Trace (final 25% as prompts).

1. The cross-covariance C₁ is smaller or larger than before? ⟨your turn: why?⟩
2. The smoothed mean’s shift from the filtered is smaller or larger? ⟨your turn⟩
3. The smoothed covariance’s reduction is smaller or larger? ⟨your turn⟩

Answers.

1. C₁ is smaller — the larger Q means the (Σ₂⁻)^{-1} is smaller, and the cross-covariance shrinks. The dynamics are looser, so the future is less informative about the past.
2. The shift is smaller — the smoother’s correction is smaller when the cross-covariance is smaller.
3. The covariance’s reduction is smaller — the smoother is closer to the filter when the dynamics are noisy, and its advantage shrinks. In the limit Q → ∞, the smoother equals the filter (the future is uninformative).

The lesson. The smoother’s advantage is largest when the dynamics are tight (Q small), so the future measurements are informative about the past, and smallest when Q is large (the future is uninformative). The tightness of the dynamics is the source of the smoother’s benefit, and it is the 0.3.1’s conditional covariance’s reduction in the temporal chain.

WE-L.4.c — Faded: the filter at T equals the smoother at T

Setup. The same trajectory, T = 5. The filter’s estimate at t = 5 is [0.5, 0.1] with covariance Σ₅. The smoother’s μ₅^s and Σ₅^s.

Question. Are μ₅^s = μ₅ and Σ₅^s = Σ₅? Why?

You must produce the intermediate steps. The setup and the answer are given.

Answer. Yes — the smoother’s initial condition is μ_T^s = μ_T and Σ_T^s = Σ_T, because at the last timestep, there is no future data to incorporate, so the smoother equals the filter. The equality is the base case of the backward recursion, and every earlier timestep is smoothed (strictly smaller covariance) when the dynamics are informative.

The lesson. The smoother’s benefit is zero at the last timestep (no future) and grows as you go backward in time (more future). At t = T − k, the smoother has k future measurements, and its benefit is the k steps’ information — for large k (far from the end), the smoother saturates (the Markov chain forgets the distant future), and its covariance converges to a limit that is smaller than the filter’s but not zero. The shape of the smoother’s covariance along the trajectory is a U (highest at the endpoints, lowest in the middle), because the endpoints have the least future data.

WE-L.4.d — Contrastive near-miss: the smoother without the cross-covariance

Correct solution. μ_t^s = μ_t + C_t(μ_{t+1}^s − μ_{t+1}⁻), with C_t = Σ_t Aᵀ (Σ_{t+1}⁻)^{-1}.

Near-miss. μ_t^s = μ_t (i.e., using the filtered estimate as the “smoothed” estimate, or equivalently C_t = 0).

Why the near-miss produces a plausible but incorrect number. The near-miss’s “smoother” is the filter, and it is a legal estimate — but it ignores the future data and its covariance is larger than the true smoother’s. The near-miss’s error is the C_t(μ_{t+1}^s − μ_{t+1}⁻) term, which is non-zero whenever the future data changes the estimate (which is always, except at t = T). The near-miss’s “smoother” is the filter, and it is not the smoother — the distinction is the entire benefit of smoothing, and its omission makes the “smoother” a misnamed filter.

The discrepancy, stated. μ_t^s = μ_t + C_t(μ_{t+1}^s − μ_{t+1}⁻) vs. μ_t^s = μ_t differ by the correction term, which is the 0.3.1 conditioning’s Σ_{ab}Σ_{bb}^{-1}(b − μ_b) for the temporal chain. The correction’s magnitude is the cross-covariance times the future’s innovation, and its direction is the cross-covariance’s. Omitting it makes the “smoother” the filter, and the error is systematic (always the filter, never the smoother), not random.

Self-explanation 8. Before reading on: in the near-miss, if the future innovations μ_{t+1}^s − μ_{t+1}⁻ are zero (the future data is perfectly predicted), does the near-miss’s “smoother” equal the true smoother?

Answer. Yes — if μ_{t+1}^s = μ_{t+1}⁻, the correction is zero, and the near-miss’s “smoother” equals the true smoother. The condition is the future data being uninformative (the prediction is exact), which happens when the process noise Q = 0 (the dynamics are deterministic) and the filter is optimal (which it is for the linear-Gaussian model with Q = 0). The near-miss is correct in the degenerate case, and wrong in the generic case where Q > 0 and the future surprises the prediction — which is the common case.

§L.4.3 — Comparative table: what the backward pass needs and what each omission costs

Design principle (P11.9). Check the smoother by its own base case before checking its benefit. The recursion has an exact identity at t = T — the smoother equals the filter — and that identity holds regardless of Q, A, or the data. A backward pass that fails it has a wiring bug, and a pass that satisfies it but shows no reduction at t < T has a cross-covariance bug; the two failures look alike from the covariance plot and are separated by this one check. Testing the identity first costs one comparison, and it distinguishes “the future is uninformative” from “the future was never read.”

§L.4.4 — Concept check

Concept check (displaces M3). A learner says: “The smoother’s gain C_t = Σ_t Aᵀ (Σ_{t+1}⁻)⁻¹ is a constant — both covariances settle once the filter converges — so I’ll compute it once from the steady state and reuse it for every timestep.” What has the learner failed to model, and what does the resulting bug look like?

Answer. The learner has failed to model that the cross-covariance is time-varying in the same sense the Kalman gain is: it is built from Σ_t, which the filter is still updating, and from Σ_{t+1}⁻, which carries the process noise injected on the way to t + 1. On a finite trajectory the filter’s covariance is still in its transient for the early steps, so C_t differs across the record — C₁ ≈ [[0.903, −0.072], [0.073, 0.795]] in WE-L.4.a, not the identity — and on any trajectory it shrinks when Q grows, which is the whole content of WE-L.4.b. A constant C_t therefore encodes one particular Q and one particular covariance, and it is wrong everywhere the actual values differ. The bug class is a gain frozen at one operating point: the smoothed mean is pulled by the right amount at the step where C_t was read and by the wrong amount elsewhere, the correction is systematically too strong near the trajectory’s ends and too weak in the middle, and the smoothed covariance no longer matches the mean it is reported with. Constancy is a consequence of convergence, not an assumption the recursion permits.

§L.4.5 — Consolidation checkpoint 4

(a) You should now be able to: (i) write the backward recursion’s three equations and name each term’s role; (ii) state the terminal condition and why it is the base case; (iii) explain the smoothed covariance’s PSD-smaller property as the future information’s contribution; (iv) state how the smoother’s advantage varies with Q, with the distance from T, and along the trajectory.

(b) Transfer probe. A learner runs the RTS smoother and finds the smoothed covariance is larger than the filtered at several timesteps in the middle of the trajectory. They check that C_t is non-zero and that the backward pass runs. What is the most likely defect, and how would you confirm it? Answer sketch: the correction C_t(Σ_{t+1}^s − Σ_{t+1}⁻)C_tᵀ is negative-semidefinite only because Σ_{t+1}^s ⪯ Σ_{t+1}⁻; the term inside the parentheses must be the smoothed minus the predicted covariance, and substituting the filtered Σ_{t+1} there — or reversing the subtraction — makes the correction positive and inflates the result. The confirmation is to compute Σ_t^s − Σ_t at the offending steps and check its sign against Σ_{t+1}^s − Σ_{t+1}⁻ at the next step: the two must have the same sign, and a mismatch localizes the substitution. The base-case check at t = T will pass either way, which is why the defect survives a casual test — the terminal step has no correction term to get wrong.

(c) Remediation rule. If you cannot do (b), return to §L.4.2 and redo WE-L.4.a, then re-attempt (b) by writing out which covariance appears in the parentheses and which sign it enters with.

Part E — Failure Taxonomy, Lab, Cheat Sheet, Exercises, Exit Test

§L.5 — Failure taxonomy

§L.6 — The lab

§L.6.1 — Task

Implement kf_predict, kf_update, kf_run, and rts_smooth; verify the KF’s one-step predict-update against the hand computation in WE-L.3.a; verify the RTS smoother’s covariance is PSD-smaller than the filter’s; then break the pipeline three ways.

§L.6.2 — Conditions

Python 3.10+, PyTorch 2.2+, CPU. Seed 42, dtype = float64. Runtime under 2 minutes. The 2D model from WE-L.3.a: A = [[1, 0.1], [0, 1]], H = [1, 0], Q = diag([0.001, 0.01]), R = 0.04. The trajectory: T = 20 steps, with a known ground-truth q₂(t) (a sinusoid) and noisy measurements.

§L.6.3 — Controls held fixed

The seed, dtype, the model matrices, and the trajectory length are fixed.

§L.6.4 — Metrics table

§L.6.5 — Pre-registered hypotheses

• H1. kf_one_step’s μ = [0.5521, 0.1148]ᵀ and Σ = [[0.00887, 0.00311], [0.00311, 0.04969]] (from WE-L.3.a). The prediction is the pair of matrices, matching to 1e-3.
• H2. kf_steady_state’s Σ is the fixed point Σ_∞ with trace(Σ_∞) < trace(Σ₀) for a larger initial Σ₀, and the K_∞ is constant after T = 20 steps. The premise matters: the filter’s fixed point here has trace = 0.0831, which is above the lab’s Σ₀ (trace = 0.0500), so the trace rises from that start. Starting from 10Σ₀ (trace = 0.5000) it falls to the same 0.0831. The prediction is the convergence to a common fixed point from both starts, and the decrease is predicted only for the larger one.
• H3. rts_cov_reduction > 0 — the smoother’s covariance is PSD-smaller at every t < T, and the sum of the traces’ reduction is positive. The prediction is the strict positivity.
• H4. rts_mean_improvement < 0 — the smoother’s estimate is closer to the truth than the filter’s, on average. This is a claim about an expectation, so one trajectory cannot test it: a single run’s gap is dominated by that run’s noise realization. Over 200 independent model-faithful trajectories the mean gap is −0.0507 ± 0.0046 (SEM), with 77% of seeds negative. The prediction is the negative ensemble mean, not a negative single-seed value. The truth must also be a sample from the model the filter assumes. Against a deterministic sinusoid — which does not obey x_{t+1} = Ax_t + w — the smoother still wins, but by a smaller margin: −0.0418 ± 0.0022 over 200 seeds, 92% negative, against −0.0507 ± 0.0046 on model-faithful trajectories. The shrinking gap is the model mismatch, not a defect of the smoother, and it is the reason the headline measurement uses a model-faithful truth.
• H5. kf_missing_Q_trace decreases monotonically to zero with Q = 0, and the filter’s estimate freezes after a few steps. The prediction is the monotonic decrease and the freeze.

§L.6.6 — The break-it sequence

1. Missing Q (silent). Set Q = 0 in the predict. Predicted symptom: no exception; the covariance shrinks to zero, the gain goes to zero, the estimate freezes, and the filter diverges from the truth. Actual: as predicted — silent.
2. Missing cross-covariance (silent). Set C_t = 0 in the smoother’s backward pass. Predicted symptom: no exception; the “smoothed” estimate equals the filter, and the covariance reduction is zero. Actual: as predicted — silent.
3. Filter-as-smoother (silent). Use μ_t (the filtered estimate) as the “smoothed” estimate without the backward pass. Predicted symptom: no exception, and the “smoothed” estimate is identical to the filter — the lab prints 0.1047 for both. Actual: silent, and on a single seed the fake smoother can score better than the real one: the lab’s seed gives 0.1047 for the filter-as-smoother against 0.1198 for the correct smoother. That inversion is the point of H4, not a contradiction of it — the smoother’s advantage is −0.0507 on the ensemble mean, and one trajectory’s gap is dominated by that trajectory’s noise. A defect detector built on a single seed’s error would clear this bug and convict the correct implementation.

All three are silent.

§L.6.7 — Acceptance criteria

§L.6.8 — What good looks like

A strong submission has five properties:

1. The one-step predict-update is computed by hand and matched against the code. The submission shows the matrices’ arithmetic and the code’s output, and the two agree.
2. The steady-state is demonstrated, not assumed. The submission runs the filter for T = 20 steps and prints the K_t trajectory, showing the convergence.
3. The smoother’s benefit is quantified, not just asserted. The submission prints trace(Σ_filter − Σ_smooth) for a few t, and the difference is positive.
4. **The break-its are predicted with the symptom (covariance freeze, zero reduction, equal estimates) stated before running, and the actuals are compared.
5. The four canonical bugs are diagnosed by name, with the mechanism stated: “missing Q → covariance decreases without bound → gain → 0 → freeze,” and so on.

§L.6.9 — Runnable lab code

# lab_0_3_3.py — Kalman filter and RTS smoother
# Requires: Python 3.10+, PyTorch 2.2+

import torch, math
torch.manual_seed(42); dtype, device = torch.float64, torch.device("cpu")

A = torch.tensor([[1., 0.1], [0., 1.]], dtype=dtype)
H = torch.tensor([[1., 0.]], dtype=dtype)  # (1, 2), observe position
Q = torch.diag(torch.tensor([0.001, 0.01], dtype=dtype))
R = torch.tensor([[0.04]], dtype=dtype)

def kf_predict(mu, Sigma, A, Q):
    mu_pred = A @ mu
    Sigma_pred = A @ Sigma @ A.T + Q
    return mu_pred, Sigma_pred

def kf_update(mu_pred, Sigma_pred, y, H, R):
    S = H @ Sigma_pred @ H.T + R
    K = Sigma_pred @ H.T @ torch.linalg.inv(S)
    mu = mu_pred + K @ (y - H @ mu_pred)
    Sigma = (torch.eye(Sigma_pred.shape[0], dtype=dtype) - K @ H) @ Sigma_pred
    return mu, Sigma, K, S

# --- Step 1: one-step check (WE-L.3.a) -------------------------------
mu0 = torch.tensor([0.5, 0.1], dtype=dtype)
Sigma0 = torch.diag(torch.tensor([0.01, 0.04], dtype=dtype))
y = torch.tensor([0.7], dtype=dtype)

mu_pred, Sigma_pred = kf_predict(mu0, Sigma0, A, Q)
print(f"predict: mu = {mu_pred.tolist()}, Sigma = {Sigma_pred.tolist()}")
mu, Sigma, K, S = kf_update(mu_pred, Sigma_pred, y, H, R)
print(f"update: mu = {mu.tolist()}, Sigma = {Sigma.tolist()}")
print(f"K = {K.tolist()}, S = {S.item():.4f}")
mu_step1 = mu.clone(); Sigma_step1 = Sigma.clone()   # H1's artifact, before the loop overwrites `mu`

# --- Step 2: ground-truth trajectory and noisy measurements ----------
# The truth must be a sample from the model the filter assumes, or the
# smoother comparison in Step 6 measures model mismatch instead of smoothing.
# A deterministic sinusoid does NOT follow x_{t+1} = A x_t + w. Measured over
# 200 seeds, the smoother still wins against a sinusoid (-0.0418 +/- 0.0022,
# 92% of seeds negative) -- but by a smaller margin than the model-faithful
# case (-0.0507 +/- 0.0046), and the gap is the mismatch, not the smoothing.
T = 20
L = torch.linalg.cholesky(Q)
x_true = torch.zeros(T, 2, dtype=dtype)
x_true[0] = torch.tensor([0.5, 0.1], dtype=dtype)   # NOT mu0: Step 1 reuses mu0
for t in range(1, T):
    x_true[t] = A @ x_true[t - 1] + L @ torch.randn(2, dtype=dtype)
y_true = x_true[:, 0:1]
y_noisy = y_true + torch.randn(T, 1, dtype=dtype) * math.sqrt(R.item())

# --- Step 3: full KF run --------------------------------------------
mu = mu0.clone(); Sigma = Sigma0.clone()
mus_filt = []; Sigmas_filt = []; mus_pred = []; Sigmas_pred = []
for t in range(T):
    mu_pred, Sigma_pred = kf_predict(mu, Sigma, A, Q)
    mus_pred.append(mu_pred.clone()); Sigmas_pred.append(Sigma_pred.clone())
    mu, Sigma, _, _ = kf_update(mu_pred, Sigma_pred, y_noisy[t], H, R)
    mus_filt.append(mu.clone()); Sigmas_filt.append(Sigma.clone())

# --- Step 4: steady-state check --------------------------------------
# H2's claim is that the trace DECREASES from a LARGER Sigma0. Starting from
# Sigma0 = diag(0.01, 0.04) the trace RISES (0.050 -> 0.083), because that
# Sigma0 is already smaller than the filter's own fixed point. Test the premise.
print(f"trace(Sigma_filt[0]) = {torch.trace(Sigmas_filt[0]).item():.4f}")
print(f"trace(Sigma_filt[-1]) = {torch.trace(Sigmas_filt[-1]).item():.4f}")

def steady_trace(S0, steps=60):
    S = S0.clone(); m = torch.zeros(2, dtype=dtype)
    for _ in range(steps):
        m, S, _, _ = kf_update(*kf_predict(m, S, A, Q), torch.zeros(1, dtype=dtype), H, R)
    return torch.trace(S).item()

big0 = 10 * Sigma0
print(f"trace(Sigma_0) = {torch.trace(Sigma0).item():.4f}  ->  steady {steady_trace(Sigma0):.4f} "
      f"(decrease: {steady_trace(Sigma0) < torch.trace(Sigma0).item()})")
print(f"trace(10*Sigma_0) = {torch.trace(big0).item():.4f}  ->  steady {steady_trace(big0):.4f} "
      f"(decrease: {steady_trace(big0) < torch.trace(big0).item()})")

# --- Step 5: RTS smoother --------------------------------------------
mus_smooth = [mu.clone() for mu in mus_filt]
Sigmas_smooth = [S.clone() for S in Sigmas_filt]
for t in range(T-2, -1, -1):
    C = Sigmas_filt[t] @ A.T @ torch.linalg.inv(Sigmas_pred[t+1])
    mus_smooth[t] = mus_filt[t] + C @ (mus_smooth[t+1] - mus_pred[t+1])
    Sigmas_smooth[t] = Sigmas_filt[t] + C @ (Sigmas_smooth[t+1] - Sigmas_pred[t+1]) @ C.T

# --- Step 6: covariance reduction and mean improvement -------------
cov_reduction = sum((torch.trace(Sigmas_filt[t]) - torch.trace(Sigmas_smooth[t])).item() for t in range(T))
err_filt = sum((mus_filt[t] - x_true[t]).norm().item() for t in range(T)) / T
err_smooth = sum((mus_smooth[t] - x_true[t]).norm().item() for t in range(T)) / T
print(f"rts_cov_reduction = {cov_reduction:.4f} (predicted > 0)")
print(f"single-seed mean improvement = {err_smooth - err_filt:+.4f} (one draw; not a test of H4)")

# H4 is a claim about the EXPECTED error, so it needs an ensemble. A single
# trajectory's smoother-vs-filter gap is dominated by that trajectory's own
# noise realization; averaging inside one run cannot see it.
def run_both(x_traj, y_traj):
    m = mu0.clone(); S = Sigma0.clone()
    mf, Sf, mp_, Sp_ = [], [], [], []
    for t in range(T):
        mpp, Spp = kf_predict(m, S, A, Q)
        mp_.append(mpp.clone()); Sp_.append(Spp.clone())
        m, S, _, _ = kf_update(mpp, Spp, y_traj[t], H, R)
        mf.append(m.clone()); Sf.append(S.clone())
    ms = [v.clone() for v in mf]
    for t in range(T - 2, -1, -1):
        C = Sf[t] @ A.T @ torch.linalg.inv(Sp_[t + 1])
        ms[t] = mf[t] + C @ (ms[t + 1] - mp_[t + 1])
    ef = sum((mf[t] - x_traj[t]).norm().item() for t in range(T)) / T
    es = sum((ms[t] - x_traj[t]).norm().item() for t in range(T)) / T
    return es - ef

gaps = []
for s in range(200):
    torch.manual_seed(1000 + s)
    xt = torch.zeros(T, 2, dtype=dtype); xt[0] = torch.tensor([0.5, 0.1], dtype=dtype)
    for t in range(1, T):
        xt[t] = A @ xt[t - 1] + L @ torch.randn(2, dtype=dtype)
    yt = xt[:, 0:1] + torch.randn(T, 1, dtype=dtype) * math.sqrt(R.item())
    gaps.append(run_both(xt, yt))
gaps = torch.tensor(gaps, dtype=dtype)
print(f"H4 ensemble over 200 seeds: mean gap = {gaps.mean():+.4f}, "
      f"SEM = {gaps.std().item() / math.sqrt(len(gaps)):.4f}, "
      f"fraction negative = {(gaps < 0).float().mean():.3f}")

# --- Step 7: break-it #1 — missing Q --------------------------------
mu = mu0.clone(); Sigma = Sigma0.clone()
Sigmas_missing_Q = []
for t in range(T):
    mu_pred, Sigma_pred = kf_predict(mu, Sigma, A, torch.zeros_like(Q))
    mu, Sigma, _, _ = kf_update(mu_pred, Sigma_pred, y_noisy[t], H, R)
    Sigmas_missing_Q.append(Sigma.clone())
print(f"[break-it #1] trace(Sigma) at t=0,5,10,19 with missing Q:")
for t in [0, 5, 10, 19]:
    print(f"  t={t}: {torch.trace(Sigmas_missing_Q[t]).item():.2e}")

# --- Step 8: break-it #2 — missing cross-covariance -----------------
mus_smooth_bad = [mu.clone() for mu in mus_filt]  # C = 0
err_smooth_bad = sum((mus_smooth_bad[t] - x_true[t]).norm().item() for t in range(T)) / T
print(f"[break-it #2] missing cross-cov mean error = {err_smooth_bad:.4f} (should equal the filter's)")
print(f"[break-it #2] correct smooth mean error = {err_smooth:.4f}")

# --- Step 9: break-it #3 — filter-as-smoother ----------------------
err_filter_as_smoother = err_filt
print(f"[break-it #3] filter-as-smoother mean error = {err_filter_as_smoother:.4f}")
print(f"[break-it #3] correct smooth mean error = {err_smooth:.4f}")

# --- Step 10: summary ------------------------------------------------
print("\n--- lab 0.3.3 summary ---")
print(f"H1: one-step mu={mu_step1.tolist()}, Sigma={[[round(v, 5) for v in r] for r in Sigma_step1.tolist()]}")
print(f"H2: trace(Sigma) 0.050 -> {steady_trace(Sigma0):.4f} from Sigma_0 (no decrease), "
      f"and 10*Sigma_0 -> {steady_trace(big0):.4f} (decrease)")
print(f"H3: rts_cov_reduction={cov_reduction:.4f} (predicted > 0)")
print(f"H4: ensemble mean gap = {gaps.mean():+.4f} +/- {gaps.std().item() / math.sqrt(len(gaps)):.4f} SEM "
      f"(predicted < 0), {(gaps < 0).float().mean():.0%} of seeds")
print(f"H5: missing-Q trace at t=19: {torch.trace(Sigmas_missing_Q[-1]).item():.2e} (predicted -> 0)")

§L.7 — The cheat sheet

§L.8 — Exercises with solutions

E 0.3.3.1 — The three problems (conceptual). For a robot arm tracking a moving target, which problem does each of the following solve: (a) the controller’s state estimate at the current time; (b) the controller’s prediction of the state at t + 0.1 for the control law; (c) the post-experiment analysis of the arm’s trajectory?

S 0.3.3.1. (a) Filtering p(x_t | y_{1:t}) — the current state, online. (b) Prediction p(x_{t+k} | y_{1:t}) — the future state, online. (c) Smoothing p(x_t | y_{1:T}) — the past state, offline. The lesson: the three problems are distinguished by their conditioning set (past vs. past+present vs. all), and the choice is the causality constraint.

E 0.3.3.2 — The KF one-step (numerical). Model: A = 1, H = 1, Q = 0.01, R = 0.01, μ = 0, Σ = 0.04, y = 1. Compute μ⁻, Σ⁻, K, μ, Σ.

S 0.3.3.2. μ⁻ = 0, Σ⁻ = 0.04 + 0.01 = 0.05. S = 0.05 + 0.01 = 0.06. K = 0.05/0.06 ≈ 0.833. μ = 0 + 0.833·(1 − 0) = 0.833. Σ = (1 − 0.833)·0.05 ≈ 0.00833. The lesson: the gain is 0.833 (the measurement is more precise than the prior, since R = 0.01 < Σ⁻ = 0.05), and the estimate moves 83% of the way to the measurement.

E 0.3.3.3 — The gain’s limits (conceptual). What is K as R → 0? As Σ⁻ → 0?

S 0.3.3.3. R → 0: S → Σ⁻, K = Σ⁻/Σ⁻ = I·H⁻¹ (or H⁻¹ if H is square), the filter fully trusts the measurement. Σ⁻ → 0: K → 0, the filter ignores the measurement and stays at the prior. The lesson: the gain is the relative precision — R → 0 means the measurement is perfect, Σ⁻ → 0 means the prior is certain — and the two limits are the extremes of the trust.

E 0.3.3.4 — The steady state (numerical). For the 1D model A = 1, Q = 0.01, R = 0.01, what is the steady-state Σ_∞ and K_∞?

S 0.3.3.4. From WE-L.2.c’s formula: Σ_∞ = (-Q + √(Q² + 4RQ))/2 = (-0.01 + √(0.0001 + 0.0004))/2 = (-0.01 + √0.0005)/2 = (-0.01 + 0.02236)/2 ≈ 0.00618. Σ_∞⁻ = Σ_∞ + Q ≈ 0.01618. K_∞ = Σ_∞⁻/(Σ_∞⁻ + R) = 0.01618/(0.01618 + 0.01) ≈ 0.618. The lesson: the steady state’s covariance is positive (not zero) and the gain is between 0 and 1, and the fixed point is the balance of the predict’s growth and the update’s reduction.

E 0.3.3.5 — The RTS cross-covariance (numerical). For A = 1, Σ_t = 0.01, Σ_{t+1}⁻ = 0.02, what is C_t?

S 0.3.3.5. C_t = Σ_t Aᵀ (Σ_{t+1}⁻)⁻¹ = 0.01·1/0.02 = 0.5. The lesson: the cross-covariance is the ratio of the filtered covariance to the predicted covariance, which is less than 1 (since the predict grows the covariance), and the smoother’s correction is C_t(μ_{t+1}^s − μ_{t+1}⁻), which is half the future innovation.

E 0.3.3.6 — The missing-Q diagnosis (diagnostic). A filter’s covariance decreases monotonically to zero, the gain goes to zero, and the estimate freezes while the state drifts. What is the bug, and what is the fix?

S 0.3.3.6. The bug is missing Q in the predict step. Q is the process noise’s covariance, the only source of new uncertainty in the predict, and its omission makes the covariance strictly decreasing. The fix is to add Q (tuned to the residual uncertainty, not the “true” process noise, since the true noise is unknowable). The lesson: Q is the filter’s acknowledgment that the model is approximate, and its omission is the overconfidence bug.

E 0.3.3.7 — The smoother’s benefit (diagnostic, transfer probe). A learner’s smoother’s covariance equals the filter’s at every timestep. What is the bug, and what is the fix?

S 0.3.3.7. The bug is C_t = 0 (or, equivalently, the backward pass is not run, or the cross-covariance’s formula is wrong). The fix is C_t = Σ_t Aᵀ (Σ_{t+1}⁻)⁻¹, and the backward pass with the correct C_t. The lesson: the smoother’s benefit is entirely in the cross-covariance term, and its omission makes the “smoother” the filter — the bug is not a small error, it is the whole smoother’s benefit.

E 0.3.3.8 — The filter vs. the smoother (constructive, transfer probe). A learner’s SLAM system uses the filter to estimate the map. The map is offline (the trajectory is recorded and processed later). What is the likely deficiency, and what is the fix?

S 0.3.3.8. The likely deficiency is acausal: the filter uses only past measurements, so the map’s features lag the true positions and their covariances are larger than necessary. The fix is the RTS smoother (or the batch SLAM back-end), which uses all measurements and produces a more accurate map with smaller covariance. The lesson: the online/offline constraint determines the estimator, and using the filter for an offline task is suboptimal (not wrong, but not the best).

§L.9 — Exit test

You are ready for Lesson 0.3.4 when you can, without notes:

1. State the three inference problems (filtering, prediction, smoothing) and their conditioning sets.
2. State the Markov and conditional-independence assumptions, and the Bayes filter’s two-step recursion.
3. State the KF’s predict (μ⁻ = Aμ, Σ⁻ = AΣAᵀ + Q) and update (S, K, μ, Σ).
4. Compute one KF predict-update by hand for a 2D state.
5. State the Kalman gain’s interpretation and its limits (R → 0 and Σ⁻ → 0).
6. State the steady-state covariance’s positivity and the gain’s convergence.
7. State the RTS smoother’s backward pass and the cross-covariance C_t.
8. Explain why the smoothed covariance is PSD-smaller than the filtered.
9. State that the smoother equals the filter at t = T, and why.
10. Diagnose missing Q (covariance shrink, freeze), missing cross-covariance (zero reduction), filter-as-smoother (equal estimates), and wrong gain (K with R instead of S) from their symptoms.
11. State the KF’s optimality’s condition (linear-Gaussian) and the nonlinear approximations (EKF, UKF, PF).
12. State the online/offline constraint and the causal/acausal estimator choice.

If any item is not answerable without notes, return to the section that introduced it: 1 to §L.1, 2 to §L.2, 3 to §L.3, 4 to §L.3.2, 5 to §L.3, 6 to §L.2.1, 7 to §L.4, 8 to §L.4.1, 9 to §L.4.2, 10 to §L.5 and the individual E’s, 11 to §L.3.1, 12 to §L.1.

§L.10 — Provenance and evidence tiers

What this lesson reproduces. Every hypothesis in §L.6.5 is executed by the lab in §L.6.9, and the text quotes the lab’s printed numbers. These claims are tier A, status R.

What this lesson does not reproduce. The sinusoid comparison quoted in H4 is not printed by the shipped lab code block, which builds a model-faithful truth only. It was measured separately over 200 seeds and is tier A, status S — reproduced, but not by the artifact the reader can run without editing. The numbers are −0.0418 ± 0.0022 (92% of seeds negative) against the model-faithful −0.0507 ± 0.0046. An earlier draft of this lesson asserted that the sinusoid makes the smoother worse than the filter (+0.13); that assertion was refuted by the measurement and is recorded here rather than deleted, because a claim about model mismatch is exactly the kind that sounds right and is not.

The H2 result is a premise check, not a disappointment. The filter’s fixed point sits above the lab’s starting covariance, so trace(Σ) rises from 0.0500 to 0.0831. The hypothesis as written predicted a decrease, and it is false for that starting value — which the lab prints rather than hides, and which the text turns into the lesson: “the steady state is a property of (A, Q, H, R), not of where you start.” The 10·Σ₀ arm shows the decrease the naive reading expects. A gate that demanded “decrease” on the original Σ₀ would have been a gate that could never pass.

On the H5 threshold. The acceptance row asks for a monotone decrease and trace < 0.02 at t = 19, measured 9.67e-3. An earlier draft of this row said < 1e-10, which the filter does not reach at t = 19 — the covariance decays geometrically and the crossing is beyond the run’s horizon. The threshold was corrected to one the run can attain, and the correction is recorded here rather than silently applied.

No external sources are cited in this lesson. The Kalman recursion and the RTS backward pass are derived in Parts B–D and executed by the lab. No paper is named. Where the lesson says “the RTS smoother,” it is naming a construction it has built, not citing a result it has not reproduced.