X0.1.4

Lesson 0.1.4 — Train vs. Eval Mode: Batch Normalization, Dropout, and the Silent Bugs They Cause in Deployment

Week 2 · Prerequisites: 0.1.1, 0.1.2, 0.1.3.

Prior results used. This lesson uses 0.1.1’s five-attribute tensor model (specifically the buffer as a persistent non-learnable tensor), 0.1.2’s .detach() / torch.no_grad() distinction (for the inference and target-computation contexts), and 0.1.3’s four-kinds-of-module-state model (parameters, buffers, submodules, plain attributes) and its canonical training loop; each is restated at the point of use.

Hands forward. This lesson hands forward one capability: the learner can look at any module in any pipeline, state which of its layers are mode-sensitive, place model.train() and model.eval() at every boundary where they are required, and diagnose a mode bug from the symptom of a model that performs differently in training and deployment — the capability that every subsequent training and evaluation lesson in the curriculum assumes.

Running case. The same 6-DOF MuJoCo arm across 8 parallel environments; the policy is now an IKNet with a batch-normalized input layer and a dropout layer in its backbone, trained at batch size 64 on 8-step pose windows and deployed on one window at a time in a real-time control loop at 50 Hz — the deployment configuration in which every mode bug in this lesson manifests. The window shape is not incidental: a policy deployed on a single 2-D observation gets a ValueError from the batch norm, and the same policy deployed on a single (C, T) window gets silence.

§L.0 — Lesson map, objectives, and misconceptions

§L.0.1 — Lesson map

§L.0.2 — Objectives

By the end of this lesson you will be able to do eight things:

1. State the single mechanism by which model.train() and model.eval() change behavior, and why a module is in training mode by default.
2. Classify any layer in a given architecture as mode-sensitive or mode-insensitive, and state which of the four kinds of module state (parameter, buffer, submodule, plain attribute) each mode-sensitive layer’s state belongs to.
3. Trace a batch-normalization forward pass in both modes, computing the per-batch statistics and the running statistics update by hand, and stating the output’s difference between the two modes.
4. Distinguish the two batch-size failures of a batch-normalized layer — the ValueError PyTorch raises when the per-channel count is 1, and the silent collapse to β when the batch is degenerate but the count is not — and state the arithmetic behind each.
5. Distinguish the two classes of mode-sensitive layer by the symptom each produces when the mode is wrong: batch-norm produces a batch-size-dependent error, dropout produces stochastic outputs.
6. Diagnose three mode bugs from their symptoms — a policy that works at training batch size and fails at deployment batch size 1, a robot that trembles at deployment, a fine-tuning run that plateaus — and name the one-line fix for each.
7. Implement the function-boundary discipline: every function that uses the model sets the mode it needs at entry, and states in a comment which mode it requires and why.
8. Produce a written “mode audit” of a training-and-deployment pipeline: every boundary at which the mode is set, the mode in which each layer executes at each boundary, and the layers whose behavior differs between the two.

§L.0.3 — Misconception inventory

Each misconception is displaced in the section named, and each is tagged in the concept check that follows it.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

This is the fourth lesson of the curriculum. The prompt specifies that warm-ups draw from two to five lessons back, which here means from Lessons 0.1.1 and 0.1.2; Lesson 0.1.3 is only one step back and is still in working memory, so it is not the source of the retrieval.

Warm-up 1 (from 0.1.1, buffers and strides). A tensor’s data_ptr() is the pointer to its storage. Two tensors with different shapes can share the same data_ptr() if one is a view of the other. Is a batch-norm running mean’s data_ptr() different from the parameter’s, and why does the answer matter? Answer: The running mean is a buffer with its own storage — its data_ptr() is different from the weight’s and bias’s — because the buffer’s values are the accumulated statistics, not a view of the parameters. The difference matters because a checkpoint that saves only the parameters’ storage would miss the running mean entirely. How this is used: §L.2’s discussion of why the running statistics must be in state_dict() and why a checkpoint that omits them silently changes the model’s behavior on reload.

Warm-up 2 (from 0.1.1, view/copy and in-place). x.add_(y) is the in-place variant of x + y. What is the difference, and why does it matter for a running-statistics update? Answer: x.add_(y) modifies x’s storage in place; x + y allocates a new tensor. The running-statistics update must be in place if it is to persist as a buffer, because running = running + ... would rebind the name to a new tensor and leave the buffer’s storage unchanged. How this is used: §L.2’s trace of the running-statistics update uses the in-place operation, and §L.2.2’s WE-L.2.b exercises the difference between rebinding and mutating.

Warm-up 3 (from 0.1.2, no_grad vs. detach). A learner calls model.eval() and torch.no_grad() in a validation loop. Which of the two changes the numerical output of the forward pass, and which changes only its memory and latency? Answer: model.eval() changes the numerical output, because it flips the training flag that mode-sensitive layers read; torch.no_grad() changes only the graph construction, so the output is numerically identical and the memory and latency are lower. How this is used: §L.4’s M5 displacement rests on this distinction, and the lab’s Step 8 demonstrates it by comparing the two.

Warm-up 4 (from 0.1.2, gradient flow). A parameter inside a mode-sensitive layer is in training mode; in eval mode, is its .grad still populated after a backward pass? Answer: It depends on whether the forward pass in eval mode is inside torch.no_grad(). If it is, no graph is built, so no .grad; if it is not, a graph is built (with eval-mode behavior), and the .grad is populated — which is usually not what the learner wants, and is why eval() and no_grad() must be called together. How this is used: §L.4’s “eval without no_grad” failure mode, and the lab’s Step 9.

Part A — The training Flag

§L.1 — Objects: the flag, its default, and its traversal

Every nn.Module has a boolean attribute self.training, set to True in the parent constructor and read by every mode-sensitive layer in its forward method. The two methods that change it are:

• module.train(mode=True) — sets self.training = mode on the module and, recursively, on every submodule, by iterating self.children(). It returns the module.
• module.eval() — equivalent to self.train(False).

The traversal is recursive, so a single model.train() at the top sets the flag on the deepest submodule. This is the mechanism, and it is the only mechanism: eval() does not clear gradients, does not disable graph construction, and does not change the parameters. It changes one boolean per module, and the mode-sensitive layers read it.

The default is True, because the parent constructor sets self.training = True and no constructor calls eval(). A learner who instantiates a model and immediately runs a forward pass is running in training mode, whether or not they intended to.

A layer is mode-sensitive if its forward method branches on self.training. The four kinds of mode-sensitive layer are:

1. Batch normalization (nn.BatchNorm1d/2d/3d) — computes per-batch statistics and updates running statistics in training; uses the running statistics in eval.
2. Dropout (nn.Dropout, nn.Dropout2d/3d, nn.AlphaDropout) — samples a mask and scales the survivors in training; is the identity in eval.
3. Instance normalization with track_running_stats=True — behaves like batch norm when the flag is set, and is mode-insensitive when it is not.
4. Any user-defined layer that reads self.training — a custom regularization, a custom sampling layer, a stochastic data-augmentation-in-the-model, or a VLA’s stochastic action head.

Everything else — nn.Linear, nn.Conv2d, nn.ReLU, nn.LayerNorm, nn.GroupNorm, nn.MultiheadAttention (with dropout=0), nn.RNN/LSTM/GRU (with dropout=0) — is mode-insensitive, because its forward does not read the flag.

§L.1.1 — Result: the flag is the only mechanism, and mode-insensitivity is a property of the layer, not of the mode

Claim. For a module m with submodules s₁, …, sₙ, m.eval() sets s_i.training = False for every i, and the output of m(x) in eval mode differs from its output in training mode if and only if at least one s_i is mode-sensitive and its two modes produce different outputs for the same input.

Proof sketch. m.eval() traverses the submodule tree and sets the flag on each module, so the flag is uniformly False after the call. The output of m(x) is the composition of the submodules’ forward methods, each of which reads its own flag; a submodule whose forward does not read the flag produces the same output in both modes, and a submodule whose forward does produces the mode-dependent output. The composition’s output therefore differs iff at least one mode-sensitive submodule’s two outputs differ for the given input.

Corollary. A model with no mode-sensitive layers has identical outputs in both modes, so a learner who omits eval() on such a model sees no difference — which is why the bug can go undetected in a small prototype (a linear model, a plain CNN without batch norm) and appears only when a mode-sensitive layer is added. The detection practice must be “call the mode at every boundary regardless of whether the current architecture needs it”, because the architecture will change.

Corollary. Mode-insensitivity is per-layer, not per-model. A model can be mode-sensitive because of one dropout layer in its transformer backbone, and every other layer in the model is mode-insensitive; the flag still changes the output, and the dropout layer is the only place the change originates.

§L.1.2 — Worked-example ladder

WE-L.1.a — Fully worked: the flag on a three-layer model

Input. The running case’s IKNet with three layers: a BatchNorm1d(3) on the input pose, a Linear(3, 64), a Dropout(0.1), and a Linear(64, 6). The learner instantiates it and inspects the flags.

Trace.

1. model = IKNet() — every submodule’s training is True, because the parent constructor set it and no constructor calls eval(). The BatchNorm1d, the Dropout, and both Linears all have training == True.
2. model.train() — sets training = True recursively; the flags are unchanged (they were already True).
3. model.eval() — sets training = False on norm, fc1, drop, fc2, and model itself.
4. model.training — False; model.norm.training — False; model.drop.training — False; etc. The flags are uniform after step 3.
5. model.train() — sets them all back to True. The flags are uniform again.

Output and interpretation. After step 3, the flags are uniform False; after step 5, uniform True. The learner should note that the flag is not a gradient flag — the parameters’ requires_grad is unaffected by eval() — and it is not a graph flag — a forward pass in eval mode still builds a graph unless torch.no_grad() is also used. The flag is only the mode selector for the mode-sensitive layers.

Self-explanation 1. Before reading on: in step 3, the Linear layers’ flags are also set to False, even though the Linear layers are mode-insensitive. Why does eval() set them, and does the flag affect their output?

Answer. Because eval() traverses every submodule and sets the flag uniformly, regardless of whether the submodule reads it. The Linear’s forward does not read self.training, so its output is identical in both modes; the flag is set but unread, and the traversal’s uniformity is a design choice (it makes the flag’s semantics simple: “the model is in eval mode”) rather than a requirement of the individual layers.

Self-explanation 2. Before reading on: in step 5, model.train() sets the flags back to True. If a learner calls model.norm.eval() after step 5, the norm layer is in eval mode and the rest of the model is in training mode. Is this a legitimate configuration, and what would it be used for?

Answer. It is a legitimate mixed-mode configuration, and it is the pattern for fine-tuning a pre-trained backbone whose batch-norm statistics must remain frozen (eval) while its other layers train (training). The norm layer’s per-batch standardization would otherwise drift away from the pre-training distribution, and freezing it (eval mode) keeps the running statistics at their pre-training values. The pattern is used in §L.4’s fine-tuning scenario, and its correctness depends on the learner deliberately choosing it and documenting the choice, not on the learner forgetting to call eval().

WE-L.1.b — Completion problem

Input. The same IKNet. The learner wants to log a validation metric every 10 steps during the training loop.

Trace (final 25% as prompts).

1. for step, (x, y) in enumerate(loader): — the training loop.
2. model.train() before the forward pass — the training phase’s mode.
3. if step % 10 == 0: — the validation block’s entry.
4. model.eval() — ⟨your turn: why is this called, and what would happen if it were omitted?⟩
5. with torch.no_grad(): — ⟨your turn: what does this do, and what would happen if it were omitted?⟩
6. The validation forward pass and the metric computation — ⟨your turn: what is the mode during the validation, and what is the flag’s state on each submodule?⟩
7. model.train() after the validation — ⟨your turn: why is this called, and what would happen if it were omitted?⟩

Answers.

1. model.eval() is called to put the mode-sensitive layers (the batch norm and the dropout) into eval mode, so the validation’s forward pass computes the deployment-mode function (frozen running statistics, no dropout). If it were omitted, the validation would compute the training-mode function (per-batch statistics, dropout active), and the validation metric would be a different quantity from the deployment metric — a systematically different number, in a direction that depends on the batch size and the dropout rate.
2. torch.no_grad() is called to disable graph construction for the validation’s forward passes, because the validation does not need gradients. If it were omitted, the forward pass would build a graph on every validation batch, wasting memory and latency, and (if the validation loss were backward-ed by mistake) populating .grad on the parameters that the training step would then apply.
3. During the validation, the mode is eval (all submodules’ flags are False), and the forward pass runs the eval-mode function for the batch norm and the identity for the dropout.
4. model.train() is called to return the mode to training for the next training iteration. If it were omitted, the next training iteration would run in eval mode, the batch norm’s running statistics would not update, the dropout would not be active, and the training would silently change its behavior — the training loss would continue to decrease (the eval-mode function is still a valid function), but the model would be training under a different regularization than intended.

The lesson. The validation block is a mode boundary: the loop must enter it (eval + no_grad) and exit it (train) explicitly. Both entries and both exits are required, and each omission has a distinct failure — the entry’s omission gives a wrong validation metric, the exit’s omission gives a training that has silently lost its regularization.

WE-L.1.c — Faded: the running case with one parameter changed

Setup. The learner wants to add a nn.LayerNorm(64) after the fc1 in the IKNet, and asks whether it changes the mode audit.

Question. Is LayerNorm mode-sensitive? Does the mode audit’s structure change? What if the learner used nn.BatchNorm1d(64) instead?

You must produce the intermediate steps. The setup and the answer are given.

Answer. LayerNorm is mode-insensitive: it computes mean and variance across the feature dimension for each example independently, so there is no batch statistic and no running average; its behavior is identical in training and eval. The mode audit’s structure does not change: the LayerNorm’s output is the same in both modes, so the model’s mode-dependence is still due to the Dropout (and any other mode-sensitive layer). If the learner used nn.BatchNorm1d(64) instead, it would be mode-sensitive, and the mode audit would need to add it to the set of layers whose behavior differs between the two modes — and the model would acquire both batch-size failures of §L.2, which LayerNorm has neither of.

The lesson. LayerNorm is the mode-insensitive alternative that avoids both the mode-sensitivity and the batch-size failures; it is the standard choice for transformers and VLA backbones precisely because it decouples the normalization from the batch, which makes deployment at batch size 1 numerically identical to training at batch size 64. The learner who swaps LayerNorm for BatchNorm in a VLA backbone has introduced a mode bug and a deployment bug, and the two are the same bug at different granularities.

WE-L.1.d — Contrastive near-miss: model.eval() alone as “the evaluation setup”

Correct solution. The evaluation function opens with model.eval(), wraps the forward passes in torch.no_grad(), and closes with model.train() if the caller is a training loop.

Near-miss. The evaluation function opens with model.eval() and does not wrap the forward passes in torch.no_grad(). The outputs are numerically correct (the mode is eval, so the batch norm and dropout behave correctly), the memory is higher than necessary, and — if the evaluation’s loss is backward-ed (which it often is not, but sometimes is, in a diagnostic that computes a gradient-based metric) — the parameters’ .grad is populated and the subsequent training step applies the evaluation’s gradient to the parameters.

Why the near-miss produces a plausible but incorrect number. The numbers are correct: the eval-mode forward pass produces the right outputs. The memory is the bug’s visible cost — the graph of every evaluation batch is built and freed, and the peak memory is the size of the evaluation’s largest graph, which can exceed the training peak and cause an OOM in a large model. The gradient contamination is the more insidious cost: a diagnostic that computes loss.backward() on the evaluation’s loss populates .grad on the parameters, and if the training loop’s zero_grad does not run between the evaluation and the training step (a plausible structure in a loop that zeroes once per iteration before the training forward), the training step applies the sum of the evaluation’s gradient and the training’s gradient.

The discrepancy, stated. model.eval() and torch.no_grad() are orthogonal operations: eval changes the function, no_grad changes the graph. The near-miss has the first and not the second, so the function is correct and the graph is built. The two-flag pattern is the correct default because neither flag implies the other, and a learner who calls only eval has fixed the numerical correctness and left the memory, latency, and gradient contamination unaddressed.

Self-explanation 3. Before reading on: in the near-miss, if the evaluation’s loss is not backward-ed, what is the observable cost of the missing no_grad?

Answer. The observable cost is the memory and latency of building and freeing a graph per evaluation batch. For a small model, this is negligible; for a VLA at batch size 1, the graph is the size of the forward pass’s activations, and the peak memory can exceed the training’s, because the evaluation’s no_grad-less forward pass retains the activations until the batch’s outputs are released. The latency cost is the graph bookkeeping, which is a fraction of the forward time but is measurable in a control loop that runs at 50 Hz.

§L.1.3 — Comparative table: mode operations and their effects

Design principle (P4.1). Two flags, two purposes, both required. In any evaluation, call model.eval() and wrap the forward pass in torch.no_grad(). The first sets the mode (which changes the function); the second sets the graph (which changes the cost). Neither substitutes for the other, and a learner who calls one has fixed half the problem.

§L.1.4 — Concept check

Concept check (displaces M1). A learner’s model trains without error, the loss decreases, and the model has a BatchNorm1d and a Dropout in it. The learner never calls model.eval(). What is wrong, and why does the decreasing loss not reveal it?

Answer. The model is in training mode for every forward pass, including the validation and the deployment. The batch norm uses per-batch statistics (so the output depends on the batch), and the dropout is active (so the output is stochastic); both are the training function, not the deployment function, and the model’s deployment behavior differs from its training behavior. The decreasing loss does not reveal it because the training loss should decrease — the model is being trained, and training mode is the mode that optimizes — and the validation metric is not compared against a deployment metric; the discrepancy is between the training and deployment numbers, and only a deployment evaluation would expose it.

§L.1.5 — Consolidation checkpoint 1

(a) You should now be able to: (i) state the single mechanism by which train() and eval() change behavior, and the default mode of a freshly constructed module; (ii) classify any layer in a given architecture as mode-sensitive or mode-insensitive, and state which of the four kinds of module state each mode-sensitive layer uses; (iii) state the two-flag pattern for evaluation (eval and no_grad) and what each flag changes.

(b) Transfer probe. A learner’s model is a transformer-based VLA with LayerNorm, MultiheadAttention(dropout=0.1), and a Dropout(0.1) at the output. The learner asks whether they need to call model.eval(). Answer sketch: Yes. The LayerNorm is mode-insensitive, but the MultiheadAttention’s dropout=0.1 is mode-sensitive (it becomes a no-op in eval), and the output Dropout(0.1) is mode-sensitive. The model’s eval-mode output differs from its training-mode output, and the difference is stochastic (dropout) rather than batch-dependent (no batch norm). The mode audit must include both dropout sites.

(c) Remediation rule. If you cannot do (b), return to §L.1.3 and re-read the MultiheadAttention row, then re-attempt (b) by asking which of the three layers reads self.training.

Part B — Batch Normalization: The Train/Eval Asymmetry

§L.2 — Objects: batch statistics, running statistics, and the two-mode function

Batch normalization for a channel c computes, from a batch of activations x ∈ ℝ^{B × C × H × W} (or ℝ^{B × C} for 1D), the per-channel mean and variance:

and normalizes:

then applies a learnable affine transform y = γ_c \hat{x} + β_c.

The four objects of the layer, and their kinds:

1. γ (weight) — a parameter, shape (C,), learnable.
2. β (bias) — a parameter, shape (C,), learnable.
3. running_mean — a buffer, shape (C,), updated by each training forward pass.
4. running_var — a buffer, shape (C,), updated by each training forward pass.

The running-statistics update, in training mode, is:

where m is the momentum (default 0.1 in PyTorch — note: PyTorch’s momentum is the new value’s weight, not the old one’s, so m=0.1 means the running statistic moves 10% toward the batch statistic per step; this is the opposite of the convention in some other frameworks, and the learner should check the framework’s convention before computing by hand).

The two-mode function:

• Training: y = γ · (x - μ_c) / sqrt(σ_c² + ε) + β, with μ_c, σ_c² from the current batch, and the running statistics updated.
• Eval: y = γ · (x - μ_running) / sqrt(σ²_running + ε) + β, with the frozen running statistics, and no update.

The two are different functions, not the same function with different inputs.

§L.2.1 — Result: batch norm’s train and eval modes compute different functions, and the difference is a batch-size dependence the framework partly refuses

Claim. For a batch-norm layer with parameters (γ, β) and buffers (μ_running, σ²_running), the training-mode output for an example x depends on the other examples in the batch through μ_c and σ_c², while the eval-mode output for the same x is a function of x alone (with μ_running, σ²_running fixed). Consequently the training-mode output is batch-size dependent. At batch size 1 the arithmetic collapses to a constant — but PyTorch does not let that arithmetic run on a 2-D input, and the distinction between where it raises and where it does not is the lesson.

Proof sketch. In training mode, μ_c and σ_c² are the batch’s per-channel mean and variance, so any change to another example in the batch changes x’s normalized output. In eval mode, μ_running and σ²_running are buffers that do not depend on the current batch, so x’s output is a fixed function of x’s value and the buffers.

Now the collapse. For a batch of size 1 the training-mode μ_c = x and σ_c² = 0, so \hat{x} = (x − x)/sqrt(0 + ε) = 0 and the layer’s output is β — a constant function, downstream receives no signal from x. That is the arithmetic, and it is exact. What PyTorch does with it depends on the shape, not on the batch size:

• A 2-D input of shape (1, C) raises. F.batch_norm calls _verify_batch_size, which rejects any input whose per-channel element count is 1 before the statistics are formed: ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 3]). The message names the real condition — more than one value per channel, not batch size greater than one.
• A 2-D input with two identical rows does not raise. Per-channel count is 2, the guard passes, σ_c² is exactly 0, and the output is exactly β. Verified: nn.BatchNorm1d(3) in training mode on [[1, 2, 3], [1, 2, 3]] returns [[0.0, 0.0, 0.0], [0.0, 0.0, 0.0]].
• A 3-D input of shape (1, C, T) does not raise. BatchNorm1d accepts (N, C, L) and normalizes over both N and L, so per-channel count is L. A single window of T = 8 timesteps passes the guard, and the layer then removes that window’s own mean across time — which is real signal, and it is gone. Measured in the lab: an input window whose per-channel means are [−0.058, −0.190, 5.087] comes out of a train-mode BatchNorm1d with means [0.158, 0.081, −0.149], equal to the layer’s trained β to four decimals, and per-channel variance γ² — the lab’s own output prints [0.545919, 0.527961, 0.797075], which are the squares of the layer’s trained γ values [0.7389, 0.7266, 0.8928].

Corollary. The training-mode batch-norm layer is a function of the batch, and the eval-mode layer is a function of the example. They coincide only in the limit of an infinite batch from a stationary distribution; in practice they differ by an amount that depends on the batch size and the data’s distribution, and the difference is systematic, not random.

Corollary. “Batch size 1 breaks batch norm” is true of the arithmetic and only sometimes true of the code. The failure is a raise when the per-channel count is 1, and a silent constant when it is greater than 1 but the batch is degenerate. A learner who has only seen the raise will conclude that their (1, C, L) deployment is safe, because it does not raise.

§L.2.2 — Worked-example ladder

WE-L.2.a — Fully worked: batch norm in train and eval, by hand

Input. A batch-norm layer with C = 1, γ = 1, β = 0, ε = 0, momentum = 0.1, and a batch of 4 activations x = [1, 2, 3, 4]. The running statistics start at μ_running = 0 and σ²_running = 1. (ε = 0 is used here for arithmetic clarity only; PyTorch rejects it at the module — see WE-L.2.b.)

Trace (training mode).

1. μ_c = (1 + 2 + 3 + 4) / 4 = 10/4 = 2.5.
2. σ_c² = [(1-2.5)² + (2-2.5)² + (3-2.5)² + (4-2.5)²] / 4 = [2.25 + 0.25 + 0.25 + 2.25] / 4 = 5/4 = 1.25.
3. \hat{x} = (x - 2.5) / sqrt(1.25 + 0) = [-1.5, -0.5, 0.5, 1.5] / 1.118 = [-1.342, -0.447, 0.447, 1.342].
4. y = 1 · \hat{x} + 0 = \hat{x}.
5. μ_running ← (1 - 0.1)·0 + 0.1·2.5 = 0.25.
6. σ²_running ← (1 - 0.1)·1 + 0.1·(5/3) = 0.9 + 0.1667 = 1.0667.

Note on step 6. The batch variance used for the normalization in step 3 is the biased one, 1.25 = 4/4; the variance used to update running_var is the unbiased one, 5/3 ≈ 1.6667 = 4/3. PyTorch uses the biased estimate to normalize the batch it is currently processing and the unbiased estimate to update the running buffer, because the buffer is meant to be an unbiased estimate of the population variance. Carrying 1.25 into the update is a common hand-calculation error and gives 1.025 instead of 1.0667.

Trace (eval mode, same layer, same batch).

1. μ_c = μ_running = 0.25 (the buffer, not the batch’s 2.5).
2. σ_c² = σ²_running = 1.0667 (the buffer, not the batch’s 1.25).
3. \hat{x} = (x - 0.25) / sqrt(1.0667) = [0.75, 1.75, 2.75, 3.75] / 1.0328 = [0.726, 1.694, 2.663, 3.631].
4. y = \hat{x}.

Output and interpretation. The training-mode output is [-1.342, -0.447, 0.447, 1.342] and the eval-mode output is [0.726, 1.694, 2.663, 3.631] — different functions of the same input. The training-mode output is centered and scaled; the eval-mode output is not, because the running statistics (0.25, 1.0667) are not the batch’s statistics (2.5, 1.25). In a converged model with many training steps, the running statistics approach the batch statistics, and the two outputs approach each other — but they never coincide exactly, and in a model that has just started training, they are far apart.

Self-explanation 4. Before reading on: in step 5, why is the running mean 0.25 and not 2.5? What would the running mean converge to after many batches with the same statistics?

Answer. The running mean is an exponential moving average with momentum m = 0.1: μ_running ← 0.9·0 + 0.1·2.5 = 0.25. It starts at 0 (the buffer’s initial value) and moves 10% toward 2.5 on each batch; after many batches with the same batch mean 2.5, the running mean converges to 2.5 (the fixed point of the moving average), and the eval-mode output approaches the training-mode output. The convergence’s time constant is 1/m = 10 batches, so after ~30 batches the running mean is within 5% of 2.5, and after ~100 batches it is within 0.3%.

Self-explanation 5. Before reading on: in step 4, the training-mode output is centered at 0 (mean 0, variance 1) and the eval-mode output is not centered. Why does the layer’s affine transform (γ = 1, β = 0) not re-center the eval-mode output?

Answer. The affine transform is a learnable shift and scale, not a standardization: γ and β are trained to optimize the task, and they learn whatever shift and scale the task requires. In this example, γ = 1, β = 0 is an untrained initialization; after training, γ and β would absorb the offset that the eval-mode output needs, and the eval-mode output would be centered (or not) according to the task’s optimum, not according to the standardization. The point is that the standardization is what differs between the modes; the affine transform is the same in both, and it is applied after the mode-dependent standardization.

WE-L.2.b — Completion problem

Input. The same batch-norm layer (γ = 1, β = 0, ε = 0, momentum = 0.1) with a batch of 1 activation x = [5].

Trace (final 25% as prompts).

1. Training mode: μ_c = 5, σ_c² = 0.
2. \hat{x} = (5 - 5) / sqrt(0 + 0) = 0/0. ⟨your turn: what does PyTorch do with this call — does it return NaN, return 0, or refuse to run?⟩
3. Running statistics update: μ_running ← (1 - 0.1)·μ_running + 0.1·5, σ²_running ← (1 - 0.1)·σ²_running + 0.1·0.
4. Eval mode: μ_c = μ_running, σ_c² = σ²_running; \hat{x} = (5 - μ_running) / sqrt(σ²_running + ε). ⟨your turn: what is y for the initial μ_running = 0, σ²_running = 1, and what is y for a converged μ_running = 5, σ²_running = 0?⟩

Answers.

1. It refuses to run, twice over, and neither refusal is the one the arithmetic suggests. The ε = 0 is rejected at the module: ValueError: batch_norm eps must be positive, but got 0.0. And even with a positive ε, a 2-D input with one value per channel is rejected before the statistics are computed: ValueError: Expected more than 1 value per channel when training. There is no NaN path here — the 0/0 never happens. To see the collapse at all you need per-channel count greater than 1, and then the answer is the arithmetic: with ε > 0, \hat{x} = 0 / sqrt(ε) = 0, so y = β. PyTorch’s two guards are what turn a documented arithmetic fact into a bug that only some shapes exhibit, and the learner who believes the NaN story will misread a ValueError as a different problem.
2. For the initial buffers (0, 1), y = (5 - 0) / sqrt(1 + 1e-5) ≈ 5.0. For converged buffers (5, 0), y = (5 - 5) / sqrt(0 + 1e-5) = 0. The initial buffers preserve the signal (the layer is nearly the identity up to γ, β), and the converged buffers collapse the single example to zero (the layer is nearly the bias). In eval mode the layer is always the well-defined (5 - μ_running) / sqrt(σ²_running + ε), a proper function of the example.

The lesson. The collapse is exactly the arithmetic (x - x)/sqrt(0 + ε) = 0, and the fix is eval mode, which uses the running statistics rather than the batch statistics. But the arithmetic is not what the code does at batch size 1 in 2-D: the framework raises. The silent version needs a per-channel count greater than 1 — two identical rows, or a (1, C, L) window — and that is the shape a deployment actually has. A lesson that teaches only the 0/0 story leaves the learner unprepared for both the raise they will meet first and the silence that follows it.

WE-L.2.c — Faded: the running case with one parameter changed

Setup. The running case’s IKNet uses BatchNorm1d(3) on the input window and is trained at batch size 64. It is deployed on a single window. The learner calls model.eval() in deployment. The model has been trained for 80 epochs, so the running statistics have converged to the training data’s statistics.

Question. In deployment (eval mode, batch size 1), what is the batch norm’s output for a pose x = [0.5, -0.2, 1.1] when the converged running statistics are μ_running = [0.0, 0.0, 0.0] and σ²_running = [1.0, 1.0, 1.0]? What is the output if the learner forgot model.eval() and the layer ran in training mode at batch size 1?

You must produce the intermediate steps. The setup and the answer are given.

Answer. In eval mode: \hat{x} = (x - μ) / sqrt(σ² + ε) = [0.5, -0.2, 1.1] / sqrt(1 + 1e-5) ≈ [0.500, -0.200, 1.100], and with γ = 1, β = 0 (an untrained initialization), y = \hat{x} — a proper standardization of the input. In training mode at batch size 1 the answer depends on the input’s shape, and the shape is the part the question must specify. For a (1, 3) input the call raises ValueError: Expected more than 1 value per channel when training; it never reaches the arithmetic. For a (1, 3, T) window with T > 1 the call succeeds, μ_c = x per channel across the window, σ_c² is the window’s own variance, and the output is the input normalized by its own time-series statistics — a function of the window, but not the function the running statistics define, and the trained β is added on top. The training-mode output is not the eval-mode output for either shape; the difference is the entire signal the layer passes downstream.

The lesson. The eval mode is not a small numerical adjustment; in training mode the batch-norm layer normalizes by the current batch and, when the batch carries no information beyond the example, erases the input. The deployment is a different model from the training model when the mode is wrong. The learner who says “the deployment model is the trained model” has assumed that the trained parameters determine the function, and in batch norm they do not — the mode is part of the function.

WE-L.2.d — Contrastive near-miss: a large-batch validation against a small-batch deployment

Correct solution. The learner calls model.eval() in both the validation and the deployment. The validation’s batch size (256) and the deployment’s batch size (1) are different, but the eval-mode batch norm’s output is batch-independent, so the same example produces the same output in both. The validation’s loss is the deployment’s loss (up to the sampling of examples).

Near-miss. The learner calls model.eval() in the validation but forgets it in the deployment. The validation’s batch norm is in eval mode (batch-independent, correct), and the deployment’s is in training mode. At a 2-D (1, C) deployment the call raises; at a (1, C, T) window it runs silently and normalizes by the window’s own statistics. The learner who met the raise has a loud bug and will fix it; the learner whose deployment shape is (1, C, T) has a silent one, sees the validation’s loss at 0.02 and the deployment’s success rate at 11%, and concludes “severe sim-to-real gap.”

Why the near-miss produces a plausible but incorrect diagnosis. The validation’s loss (0.02) and the deployment’s success rate (11%) are both real numbers from the learner’s runs, and the gap is consistent with a sim-to-real gap, a distribution shift, or a data issue. The natural diagnosis is “the real environment is different from the simulation,” and the natural response is to collect more data, randomize the simulation, or fine-tune on the real environment — none of which addresses the mode bug. The mode bug is invisible in the validation (the validation is correct) and in the training (the training is in training mode, by design), and only the deployment is affected, which is the one place the learner cannot easily A/B test against a known-correct baseline.

The discrepancy, stated. The correct solution’s validation and deployment both use eval mode; the near-miss uses eval in one and training in the other. The difference is a single line in the deployment code, and the symptom is a 100×-looking performance gap that is entirely attributable to that line. The learner’s diagnosis is wrong not because the sim-to-real gap is unimportant, but because the gap they are seeing is not the sim-to-real gap — it is the mode gap, and the two are distinguished by running the deployment in eval mode and observing that the success rate jumps to the validation’s level.

Self-explanation 6. Before reading on: in the near-miss, what would the deployment’s batch-norm output be for a batch of 1 example, and how does the output differ from the validation’s batch of 256? What would the deployment’s success rate be if the learner used batch size 256 at deployment?

Answer. In training mode the layer normalizes by the current batch’s statistics. At batch size 1 in 2-D the call raises (Expected more than 1 value per channel when training) — it does not return a constant, because PyTorch refuses the degenerate batch outright. At a per-channel count greater than 1 the call succeeds and the output is γ · (x − μ_c)/sqrt(σ_c² + ε) + β with μ_c, σ_c² from the batch; for a degenerate batch (identical examples, or a single window) that reduces to a constant equal to β, and for a normal batch it is close to the eval-mode function. At batch size 256 the batch’s statistics are close to the running statistics, so a 256-example deployment in training mode works — the success rate would be close to the validation’s. This is why the bug is batch-size dependent: a deployment at a large batch size in training mode accidentally approximates the correct behavior, and a deployment at a small one either raises or collapses. The bug is latent at large batch sizes and catastrophic at small ones, and the deployment’s shape is the variable that determines which failure you get.

§L.2.3 — Comparative table: the batch norm’s two modes

Design principle (P4.2). Batch norm’s mode is part of the model’s function, not part of the training setup. A batch-norm layer’s function is different in training and eval, and the trained parameters do not determine which function the layer computes — the mode does. The deployment code must set the eval mode explicitly, and a deployment that omits it is deploying a different model than the one that was validated.

§L.2.4 — Concept check

Concept check (displaces M2, M3). A learner says: “Batch norm standardizes the input, and the running statistics are just a cache of the batch statistics.” Give the two reasons this model is wrong, and name the mode in which each reason matters.

Answer. First, batch norm is per-batch standardization in training mode (the output depends on the batch’s composition), and running-statistics standardization in eval mode (the output is a function of the example alone) — the two are different functions, not the same function with a cache. Second, the running statistics are an exponential moving average of the batch statistics, not a cache: they converge to the batch statistics only in the limit of a stationary distribution, and they lag by a factor of 1/m batches, so a model that has just started training has running statistics far from the batch statistics, and a model that is fine-tuning on a new distribution has running statistics that are frozen at the pre-training distribution unless the layer is in training mode (which is §L.4’s fine-tuning topic). The first reason matters in eval mode; the second matters in fine-tuning and in resuming from a checkpoint.

§L.2.5 — Consolidation checkpoint 2

(a) You should now be able to: (i) trace a batch-norm forward pass in both modes by hand, including the running-statistics update; (ii) state the two batch-size failures — the raise at per-channel count 1 and the silent collapse to β at a degenerate batch — and the role ε plays in neither of them being a NaN; (iii) state why a validation at batch size 256 and a deployment at batch size 1 agree in eval mode and disagree in training mode; (iv) state why a model in training mode is a different function from the same model in eval mode, and why the trained parameters do not determine which function is computed.

(b) Transfer probe. A learner’s VLA is trained at batch size 32 with BatchNorm1d in its image patch embedding. They validate at batch size 32 with model.eval(), achieve 0.02 validation loss, and deploy at batch size 1 with model.eval(), achieving 0.02 deployment loss. The deployment’s success rate is 0.90. A second learner’s identical setup achieves 0.02 validation loss and 0.90 deployment loss — they forgot model.eval() in the deployment. Which learner’s deployment is the trained model, and what is the difference in the model’s function? Answer sketch: the first learner’s deployment is the trained model, because the eval mode’s function is the one the validation measured; the second learner’s deployment is a different function — the training-mode batch norm — and whether it collapses to a constant depends on the shape: a 2-D (1, C) patch embedding would raise, while a (1, C, L) one silently normalizes each patch by its own statistics. Either way the downstream layers see a different function from the validated one. The two deployments have the same trained parameters and different functions, and the mode is the variable that distinguishes them.

(c) Remediation rule. If you cannot do (b), return to §L.2.2 and redo WE-L.2.c, then re-attempt (b) by asking what the batch norm’s output is at batch size 1 in each mode.

Part C — Dropout: Stochastic Masking and the Two Modes

§L.3 — Objects: the mask, the inverted scaling, and the no-op

Dropout for an input x and rate p:

• Training mode: sample a binary mask m with m_i ~ Bernoulli(1 - p), set y_i = m_i · x_i / (1 - p). The scaling 1/(1-p) is inverted dropout; it restores the expected value E[y_i] = x_i so that eval mode needs no rescaling.
• Eval mode: y = x — the identity, no mask, no scaling.

The two objects are the mask (a per-forward-pass tensor) and the rate (a hyperparameter, not a parameter, not a buffer — a plain attribute). There are no learned parameters and no buffers in a Dropout layer; its entire state is the rate and the training flag.

The statistical content of dropout is:

• The mask is a random variable; the output is a random variable.
• The expected value of the output is the input: E[y_i] = x_i (the inverted scaling’s purpose).
• The variance of the output is Var[y_i] = x_i² · p/(1-p) (a per-element variance that grows with p).

The Bayesian interpretation (MC dropout, §L.3.4): keeping dropout active at inference samples from an approximate posterior over the network’s functions, and the variance across samples is an estimate of the model’s predictive uncertainty.

§L.3.1 — Result: dropout is the identity in eval, and a stochastic mask in training; the mode controls which function the layer computes

Claim. For a Dropout(p) layer with 0 < p < 1, the training-mode output is a random variable whose distribution depends on p and the input, and the eval-mode output is the input itself (deterministic, identity). The training-mode output is unbiased for the input (E[y] = x) but has variance proportional to p/(1-p), while the eval-mode output has zero variance.

Proof sketch. The training-mode output is y_i = m_i · x_i / (1-p) with m_i ~ Bernoulli(1-p). Its expectation is E[y_i] = E[m_i] · x_i / (1-p) = (1-p) · x_i / (1-p) = x_i. Its variance is Var[y_i] = Var[m_i] · (x_i/(1-p))² = p(1-p) · x_i² / (1-p)² = x_i² · p/(1-p). The eval-mode output is the input, whose expectation is x_i and variance is 0. So the two modes have the same expectation and different variance — the training mode is a noisy version of the eval mode, with the noise’s variance controlled by p.

Corollary. A model evaluated in training mode has a stochastic output, and the loss over a set of examples is a random variable. The variance of the loss decreases with the number of examples in the evaluation batch (the per-example noise averages out), so a large-batch validation in training mode is close to the eval-mode validation, and a small-batch (or batch-size-1) validation in training mode is noisy. This is the dropout analogue of batch norm’s batch-size dependence, but the mechanism is different: dropout’s noise averages out with batch size, and batch norm’s batch dependence changes the function with batch size.

Corollary. A deployment at batch size 1 with dropout in training mode produces a different action for the same observation on each call — the robot trembles — and the tremor’s magnitude is proportional to the action’s magnitude and p. A deployment with dropout in eval mode is deterministic and does not tremble.

§L.3.2 — Worked-example ladder

WE-L.3.a — Fully worked: dropout in train and eval, with a concrete mask

Input. A Dropout(p = 0.5) layer with input x = [2.0, 4.0, 6.0]. The training-mode mask is m = [1, 0, 1].

Trace (training mode).

1. m = [1, 0, 1] — m_1 = 1 (keep), m_2 = 0 (drop), m_3 = 1 (keep).
2. y_1 = 1 · 2.0 / (1 - 0.5) = 2.0 / 0.5 = 4.0.
3. y_2 = 0 · 4.0 / 0.5 = 0.0.
4. y_3 = 1 · 6.0 / 0.5 = 6.0 / 0.5 = 12.0.
5. y = [4.0, 0.0, 12.0] — the output is not the input; it is a scaled, masked version.

Trace (eval mode, same layer, same input).

1. y = [2.0, 4.0, 6.0] — the identity.

Output and interpretation. The training-mode output has two of the three inputs (the dropped one is 0) and is scaled by 2 to preserve the expectation. The eval-mode output is the input. The expectation of the training-mode output over the mask distribution is [2.0, 4.0, 6.0] (the eval-mode output), so the two modes have the same expectation and different variance — the training mode’s variance is the mechanism that regularizes the model, and the eval mode’s zero variance is what makes deployment deterministic.

Self-explanation 7. Before reading on: in step 2, why is the scaling 1/(1-p) = 2 rather than 1? What would happen to the model’s downstream layers if the scaling were 1?

Answer. The scaling is 1/(1-p) = 2 because the mask keeps each input with probability 1-p = 0.5, so without scaling the expected output would be 0.5 · x, which is half the input’s expected value; the scaling restores the expected value to x. If the scaling were 1, the training-mode output’s expectation would be 0.5 · x, and every downstream layer would see a systematically attenuated signal — the model could compensate by doubling its downstream weights, but the compensation is not the intended behavior, and the model’s effective function after training would be 2× the intended one, which is wrong at deployment (where dropout is off and the doubling is no longer compensated). The inverted scaling avoids this by making the training-mode expectation equal the eval-mode output, so no compensation is needed.

WE-L.3.b — Completion problem

Input. The same Dropout(p = 0.5) layer, with the mask m = [0, 1, 0].

Trace (final 25% as prompts).

1. y_1 = 0 · 2.0 / 0.5 = 0.0.
2. y_2 = 1 · 4.0 / 0.5 = 8.0. ⟨your turn: what is y_3?⟩
3. y = [0.0, 8.0, 0.0]. ⟨your turn: what is the average of y over all four possible masks (000, 001, 010, 011, 100, 101, 110, 111)?⟩
4. Eval mode: y = x = [2.0, 4.0, 6.0]. ⟨your turn: does the training-mode average equal the eval-mode output? Why?⟩

Answers.

1. y_3 = 0 · 6.0 / 0.5 = 0.0.
2. The average over the 8 masks is [2.0, 4.0, 6.0] — the eval-mode output — because each element is independently kept with probability 0.5 and scaled by 2, so each element’s average is its input value.
3. Yes, the training-mode average equals the eval-mode output, because the scaling 1/(1-p) is exactly what makes the expectation match. This is the purpose of inverted dropout, and it is why eval mode needs no rescaling.

The lesson. The training-mode output is a random variable whose expectation is the eval-mode output; the mode does not change the function’s expectation, it changes its variance. The learner who reports “the training output is different from the eval output” is correct — the outputs differ per sample — and the difference is the noise, not a bias. The eval-mode output is the noise-free mean of the training-mode outputs.

WE-L.3.c — Faded: the running case with one parameter changed

Setup. The running case’s IKNet has a Dropout(p = 0.1) in its backbone and is deployed at batch size 1 in a real-time control loop at 50 Hz. The learner calls model.eval() in deployment.

Question. What is the difference between the deployment’s output and the same model’s output if the learner forgot model.eval() and ran the deployment in training mode? Estimate the tremor’s magnitude relative to the action’s magnitude, and state whether the tremor is deterministic or stochastic.

You must produce the intermediate steps. The setup and the answer are given.

Answer. With model.eval(), the dropout is the identity, and the deployment’s action is a deterministic function of the observation. Without model.eval(), the dropout samples a mask on every call, so the action changes between consecutive calls with the same observation — the robot’s end-effector trembles, with the tremor’s magnitude proportional to the action’s magnitude and p = 0.1. The tremor is stochastic (a different mask on each call), so it is not a constant bias that can be calibrated out — it is a variance that must be removed by setting the mode.

The lesson. The dropout mode bug manifests as a stochastic symptom, not a systematic bias, so the learner who averages the robot’s behavior over many trials sees the mean behavior as correct (the expectation matches the eval-mode output) and only the variance as inflated. The averaged behavior looks fine; the instantaneous behavior is wrong. The learner who diagnoses “the controller gains are wrong” or “the robot has a sensor noise issue” has attributed the variance to the wrong source; the mode is the source.

WE-L.3.d — Contrastive near-miss: MC dropout without documentation

Correct solution. The learner wants uncertainty estimates from a dropout-regularized VLA, so they deliberately keep the dropout layers in training mode while the rest of the model is in eval. The pattern is:

model.eval()                                    # all layers in eval mode...
for m in model.modules():
    if isinstance(m, nn.Dropout):
        m.train()                              # ...except the dropout layers
with torch.no_grad():
    actions = [model(obs) for _ in range(20)]   # 20 samples from the approximate posterior
action_mean = torch.stack(actions).mean(0)
action_std = torch.stack(actions).std(0) / (20 ** 0.5)   # the posterior's std: std(0) estimates sigma, not sigma/sqrt(n)

The action mean is the deployment action, and the action std is the uncertainty — used by the downstream safety layer to decide whether to act or to request human assistance. std(0) alone estimates the per-sample noise σ; dividing by √n converts it to the std of the mean, which is the quantity an uncertainty threshold should be calibrated to. The mixed mode is documented, deliberate, and tested.

Near-miss. The learner’s model is in training mode throughout (they never call model.eval()), and the deployment takes one forward pass per observation. The output is a single sample from the dropout distribution, not the mean, and its variance is the full dropout variance. The learner interprets the output as “the model’s best guess” and the variance as “the model’s uncertainty” — but the uncertainty is only the dropout variance, it is not calibrated, and the mean is not what the deployment uses (the single sample is biased, because the inverted scaling’s mean is zero-mean-noise around the true mean, and the single sample has that noise).

Why the near-miss produces a plausible but incorrect number. The single-sample output has the correct expectation (the true mean), so an averaged deployment’s behavior is correct — but the instantaneous behavior is noisy, and the “uncertainty” the learner reports is the single-sample noise, not the model’s uncertainty. A downstream safety layer that uses the single-sample noise as its uncertainty estimate is using a noise estimate, not a posterior estimate, and the safety threshold is miscalibrated. The learner who compares the reported uncertainty against a held-out set and finds it uninformative (the uncertainty does not correlate with error) has found a symptom that is consistent with several causes (a bad model, a bad uncertainty estimator, a bad calibration), and the actual cause is the single-sample interpretation.

The discrepancy, stated. torch.stack(actions).std(0) estimates the per-sample std σ — the spread of individual stochastic forward passes around their mean — not the std of the mean. The std of the mean is σ/√n, so the posterior estimate is torch.stack(actions).std(0) / √n. The two differ by a factor of √n — a factor of ~4.5 for n = 20 — and the uncorrected std(0) is 4.5× larger than the posterior’s, so a safety threshold calibrated to the posterior is triggered too often. Measured: with σ = 1 and n = 20, std(0) averages 0.99 across replicates against σ/√n = 0.224. The learner who reports std(0) as “the model’s uncertainty” has reported the dropout noise, which is a per-sample quantity and does not shrink with more samples; the posterior’s uncertainty is the one that does.

Self-explanation 8. Before reading on: what does torch.stack(actions).std(0) actually estimate, and what extra operation turns it into the posterior’s std?

Answer. Stacking n forward passes gives n values at each output position, and std(0) is the standard deviation of those n values around their own mean — which is an estimate of the per-sample std σ, the spread of one stochastic pass. It is not the std of the mean. The mean of n independent draws has variance σ²/n, so its std is σ/√n, and the posterior estimate is therefore std(0) / √n. The distinction matters because the two quantities behave differently as n grows: σ is a property of the network and does not shrink, while σ/√n shrinks as 1/√n and is the quantity that represents “how well do we know the mean”. A deployment that thresholds on σ is thresholding on the network’s noise level, which is a constant of the model, not a statement about confidence in this particular action.

§L.3.3 — Comparative table: the dropout modes and their uses

Design principle (P4.3). If you keep dropout on, average the samples — and divide by √n before calling the result an uncertainty. torch.stack(actions).std(0) estimates the per-sample std σ, not the std of the mean; the posterior estimate is std(0) / √n. A deployment that thresholds on std(0) is thresholding on the network’s noise level, which does not shrink as n grows, so the threshold is miscalibrated by a factor of √n. If the deployment’s task is a point action (no uncertainty), set eval(); if the task is uncertainty-aware, sample n ≥ 10, average, and divide.

§L.3.4 — Concept check

Concept check (displaces M4). A learner says: “Dropout is a regularizer, so it is only active during training, and model.eval() is always the right call.” Name the use case in which model.eval() is not the right call for the dropout layers, and state the one-line pattern that implements it.

Answer. The use case is MC dropout for uncertainty estimation, where the dropout layers must remain in training mode while the rest of the model is in eval mode; the pattern is model.eval() followed by for m in model.modules(): if isinstance(m, nn.Dropout): m.train(). The eval() sets the base mode, and the loop re-enables only the dropout layers, so the batch norm (if any) is in eval and the dropout is in training. The deliberate mixed mode is the difference between a correct uncertainty estimate and the “single-sample noise” near-miss, and it must be documented in the code.

§L.3.5 — Consolidation checkpoint 3

(a) You should now be able to: (i) state the training and eval functions of dropout, the inverted scaling’s purpose, and the expectation/variance relationship between them; (ii) trace a dropout forward pass with a concrete mask and compute the output’s expected value across masks; (iii) state the MC dropout pattern and the difference between the posterior’s std (σ/√n) and the per-sample’s std (σ); (iv) diagnose a “trembling robot” as a dropout-in-training-mode bug from the symptom of stochastic outputs for identical inputs.

(b) Transfer probe. A learner’s policy has a Dropout(0.2) and is deployed at 50 Hz. The robot’s end-effector position at a fixed target oscillates with a different amplitude on each trial — sometimes 1 mm, sometimes 5 mm — and the average amplitude is 2 mm. The learner has checked the controller gains, the sensor noise, and the calibration. What is the most likely cause, and how would the learner confirm it in a single experiment? Answer sketch: the cause is dropout in training mode at deployment; the amplitude varies because the masks vary, and the average is the RMS of the dropout noise (proportional to p = 0.2 and the action magnitude). The confirming experiment is to call model.eval() and observe the oscillation disappear — the amplitude becomes zero (or the deterministic value, if other noise sources remain). If the oscillation disappears, the cause is the mode; if it does not, the cause is elsewhere, and the experiment has ruled out the mode.

(c) Remediation rule. If you cannot do (b), return to §L.3.2 and redo WE-L.3.c, then re-attempt (b) by asking whether the symptom is deterministic (a controller or calibration issue) or stochastic (a sampling issue, which dropout is).

Part D — Mixed Modes and the Function-Boundary Discipline

§L.4 — Objects: the three legitimate mixed-mode scenarios

The three scenarios in which the learner deliberately wants part of the model in training and part in eval are:

1. Fine-tuning a pre-trained backbone with frozen batch-norm statistics. The backbone’s batch norm must not update its running statistics (eval mode), while the backbone’s other layers (and the new head) train (training mode). The pattern is model.train(); model.backbone.eval(), or the equivalent by iterating the backbone’s batch-norm layers.
2. MC dropout for uncertainty estimation. The dropout layers are in training mode while the rest is in eval, as §L.3.4 describes.
3. A stochastic action head in an otherwise-deterministic policy. The head samples from a distribution while the perception backbone is deterministic; the head’s sampling layer reads self.training and the backbone’s layers do not.

The three are legitimate because each has a reason, each is deliberate, and each must be documented and tested. The illegitimate mixed mode is the accidental one — a learner who calls model.eval() on the top-level model but not on a submodule they added later, or a learner who calls model.train() on one path but not another, and the model’s mode is a function of the call order rather than of the design.

§L.4.1 — Result: the mode should be set at every function boundary, not once at the top of the script

Claim. For a pipeline with f functions that use the model (train_step, evaluate, deploy, compute_uncertainty), the mode discipline is correct if and only if each function sets the mode it needs at its entry and restores the mode it found at its exit (or leaves it in a defined state for the caller), and not if the mode is set once at the top of the script and assumed to be uniform thereafter.

Proof sketch. The top-level call order determines the mode at each function’s entry only if the functions do not change it; but functions that need a different mode must change it, and the change persists after the function returns unless it is undone. So the mode at a function’s entry is determined by the last function that set it, not by the top-level script’s initial setting. The discipline that makes the mode a function of the design rather than a function of the call order is to set it at each entry: evaluate sets eval, train_step sets train, compute_uncertainty sets eval + dropout-training, etc. The pattern is the analogue of local reasoning in software: each function’s behavior is determined by its own code, not by its caller’s state.

Corollary. The discipline’s implementation is to open each function with the mode set and to not rely on the caller’s mode. A train_step that opens with model.train() is correct whether the caller left the model in eval or in training; an evaluate that opens with model.eval() is correct regardless of the caller’s state. The pattern is idempotent (setting the mode twice is the same as setting it once), so the cost is zero and the benefit is the elimination of the call-order dependence.

§L.4.2 — Worked-example ladder

WE-L.4.a — Fully worked: the function-boundary discipline

Input. A training pipeline with three functions: train_step(model, optimizer, batch) (one training iteration), evaluate(model, loader) (a validation pass), and deploy(model, obs) (one deployment inference).

Trace (function bodies).

1. train_step opens with model.train() — it needs training mode (dropout active, batch norm per-batch). The function’s forward pass and backward pass are in training mode.
2. evaluate opens with model.eval(), then with torch.no_grad():, then the loop. The function’s forward passes are in eval mode (dropout off, batch norm frozen), and the graph is not built.
3. evaluate ends with model.train() — it restores the training mode, so the caller’s training loop can continue without a mode change. (This is the restore step; the alternative is to leave the mode at eval and require the caller to restore, which is more error-prone.)
4. deploy opens with model.eval(), then with torch.no_grad():, then the forward pass. The deployment’s mode is eval, regardless of the caller’s state (the deployment is a separate program in most cases, but the discipline applies regardless).
5. The training loop calls train_step and periodically evaluate; each function sets its own mode at entry, so the loop’s correctness does not depend on the loop’s ordering of calls.

Output and interpretation. The three functions each set the mode they need at entry, and evaluate restores the training mode at exit. The correctness is local to each function, and the pipeline’s behavior is the composition of the functions’ correct behaviors, regardless of the call order. The learner who sets the mode only at the top of the script has made the pipeline’s behavior global — a change in the call order changes the behavior — and the bug is a call-order bug, not a function bug, which is much harder to diagnose.

Self-explanation 9. Before reading on: in step 3, why does evaluate end with model.train(), and what would happen if it did not?

Answer. Because the function changed the mode (to eval) and the caller (train_step’s loop) expects the training mode to persist. If evaluate did not restore the training mode, the next train_step call would open with model.train() anyway (because the function-boundary discipline makes each function set its own mode), so the restoration is redundant — but it is defensive, and it makes the function’s effect idempotent from the caller’s perspective. The redundancy is the point: the function’s correctness does not depend on the caller’s restoration, and the restoration makes the pipeline’s behavior robust to a caller that does depend on it (a caller that does not use the function-boundary discipline).

WE-L.4.b — Completion problem

Input. A fine-tuning scenario: a pre-trained backbone with batch norm, a new action head, and the learner wants the backbone’s batch norm frozen (eval mode) while the backbone’s other layers and the head train.

Trace (final 25% as prompts).

1. model.train() — ⟨your turn: what does this do to all submodules?⟩
2. for m in model.backbone.modules(): if isinstance(m, nn.BatchNorm1d): m.eval() — ⟨your turn: what does this do to the backbone’s batch-norm layers, and what does it leave the other layers in?⟩
3. The forward pass and backward pass — ⟨your turn: which layers update their running statistics, and which do not?⟩
4. The deployment: model.eval() — ⟨your turn: what changes at deployment, and does the frozen batch norm’s mode differ from the deployment’s?⟩

Answers.

1. model.train() sets training = True on every submodule — the backbone’s batch norm, the backbone’s other layers, and the head.
2. The loop sets training = False on the backbone’s batch-norm layers only; the backbone’s other layers and the head remain in training mode. The result is the desired mixed mode.
3. The backbone’s batch-norm layers are in eval mode, so they do not update their running statistics; the backbone’s other layers and the head are in training mode, so their parameters train. The running statistics stay at their pre-training values, which is the freeze the learner intended.
4. At deployment, model.eval() sets all submodules to eval, including the backbone’s batch norm (already eval, idempotent) and the head (now eval, which is what the deployment needs). The frozen batch norm’s mode is the same in fine-tuning and in deployment (eval), so the fine-tuning’s batch-norm behavior is the same as the deployment’s — which is the point of freezing it. A model whose fine-tuning batch norm is in training mode would have different batch-norm behavior at fine-tuning and at deployment, and the deployment would be a different function.

The lesson. The frozen batch norm is eval in fine-tuning and eval in deployment, so the layer’s mode is the same in both — the freeze is what makes the fine-tuning’s mode match the deployment’s mode. A learner who does not freeze the backbone’s batch norm has the layer in training mode during fine-tuning (updating its running statistics on the new data) and in eval mode during deployment (using the updated running statistics) — the two are the same function only if the running statistics converge to the new data’s statistics, which they do over many fine-tuning steps but do not in the first few, so the fine-tuning’s early behavior is a different function from the deployment’s. The freeze is the choice that makes the two match, and it is the right choice when the new data is small (the running statistics would overfit to the small batch) and the wrong choice when the new data is large and the pre-training statistics are wrong (in which case the running statistics should adapt).

WE-L.4.c — Faded: the running case with one parameter changed

Setup. The running case’s IKNet is fine-tuned on a new arm with a slightly different kinematic chain, and the learner does not freeze the backbone’s batch norm (they keep model.train() throughout).

Question. What happens to the batch norm’s running statistics during fine-tuning, and how does the deployment’s behavior compare to the fine-tuning’s? What is the right choice — freeze or not — for a 100-example fine-tuning set and a 10,000-example fine-tuning set?

You must produce the intermediate steps. The setup and the answer are given.

Answer. Without the freeze, the batch norm’s running statistics update on the fine-tuning data, moving toward the new arm’s statistics. With a 100-example fine-tuning set, the running statistics are estimated from very few examples per batch (a batch of 64 has 64 examples from a 100-example set, so the statistics are noisy and non-representative), and the deployed model’s eval-mode behavior is the noisy running statistics, not the true new-arm statistics — the deployment is worse than if the pre-training statistics were kept (which at least reflected the pre-training arm, which is close). The right choice for 100 examples is freeze (keep the pre-training statistics, which are close enough to the new arm’s). For 10,000 examples, the running statistics are estimated from many examples and are representative of the new arm; the right choice is not freeze (let the statistics adapt to the new arm, which is the arm the deployment will use). The choice is a bias-variance trade-off in the running statistics: freeze → low variance, higher bias (the pre-training statistics); no freeze → higher variance (the fine-tuning statistics’ noise), lower bias (the new arm’s statistics). The rule of thumb is: freeze when the fine-tuning set is small relative to the batch size, and do not freeze when it is large.

The lesson. The fine-tuning freeze is a statistical choice, not a convention; the right choice depends on the fine-tuning set’s size, the pre-training distribution’s relevance, and the batch size’s relationship to the fine-tuning set’s size. The learner who freezes “because it is the convention” has not evaluated the trade-off, and the learner who does not freeze “because it is simpler” has not either. The choice is explicit and stated in the code, with its rationale.

WE-L.4.d — Contrastive near-miss: the accidental mixed mode

Correct solution. The learner’s model has a BatchNorm1d in the backbone and a Dropout in the head. They fine-tune with model.train() and deploy with model.eval(). The modes are uniform in each phase, and the deployment’s function matches the training’s function (for the layers whose modes are the same) and the validation’s function (for all layers).

Near-miss. The learner adds a new Dropout to the head after the first training run and forgets to check that the fine-tuning’s mode is uniform. The training loop’s model.train() is called once, before the loop, but a diagnostic function called in the middle (an evaluate that does not restore the mode) leaves the model in eval mode for the rest of the training. The first training run (before the diagnostic) is in training mode; the second (after) is in eval mode. The model’s behavior changes mid-training, and the change is invisible in the loss curve (both modes’ losses decrease), but the dropout in the head is off in the second run, so the regularization is lost, and the batch norm in the backbone is frozen, so its statistics stop updating.

Why the near-miss produces a plausible but incorrect number. The loss still decreases in both runs (the eval-mode function is still a valid function that the optimizer can fit), and the validation loss is computed in eval mode (which is what the diagnostic set), so the validation is correct and the training is half-correct — the first half in training mode, the second in eval. The model’s final state is a mixture of the two modes’ optimizations, and the deployed model (in eval mode) is the model that the second half’s optimization produced, which is a different model than the one the first half’s loss measured. The learner who compares the final validation loss to the initial validation loss sees the expected decrease and concludes the training was fine, without noticing that the middle of the training changed mode.

The discrepancy, stated. The correct solution’s mode is uniform throughout each phase; the near-miss’s mode changes in the middle, because a diagnostic left it in eval. The change is invisible in the loss curve and in the final validation (both are in the mode the diagnostic set), and the only way to detect it is to log the mode at each phase’s entry, or to enforce the function-boundary discipline so that each function sets its own mode and the mode cannot be left in the wrong state by a diagnostic. The near-miss is the “accidental mixed mode” that the discipline prevents.

Self-explanation 10. Before reading on: in the near-miss, what is the one line the diagnostic function should have added to prevent the mode leak, and why does that line make the leak impossible?

Answer. The diagnostic should have restored the mode at its exit — model.train() at the end of the diagnostic — or, better, the training loop should have set the mode at each iteration’s entry (model.train() inside the loop, not once before it). The first fix restores the mode for the next iteration, and the leak is one iteration long; the second fix makes the mode idempotent per iteration, and the leak is zero iterations long. The function-boundary discipline is the second fix, and it makes the leak structurally impossible because the mode is set by the function that needs it, not by the function that ran before.

§L.4.3 — Comparative table: mode discipline patterns

Design principle (P4.4). Set the mode at the function boundary, not at the script’s top. The mode is a per-function concern, not a per-script concern; a function that uses the model knows which mode it needs, and setting the mode at the function’s entry makes the function’s behavior local and independent of the caller’s state. The pattern is idempotent, so it costs nothing, and it makes the accidental mixed mode structurally impossible — the mode cannot be left in the wrong state by a diagnostic, because the next function sets it.

§L.4.4 — Concept check

Concept check (displaces M5). A learner says: “model.eval() and torch.no_grad() are both for evaluation, so I call one of them.” Name the two things each changes, and the symptom of calling only model.eval().

Answer. model.eval() changes the training flag, which changes the function the mode-sensitive layers compute (batch norm uses running statistics, dropout is the identity). torch.no_grad() changes graph construction, which changes the memory and latency of the forward pass and (if the evaluation’s loss is backward-ed) whether the parameters’ .grad is populated. Calling only model.eval() produces the correct numerical output and higher memory (the graph is built), and (if the evaluation’s loss is backward-ed) gradient contamination (the evaluation’s gradient is applied to the parameters). The two are orthogonal, and both are required.

§L.4.5 — Consolidation checkpoint 4

(a) You should now be able to: (i) state the three legitimate mixed-mode scenarios and the pattern for each; (ii) implement the function-boundary discipline, with each function setting its own mode at entry; (iii) state why the function-boundary discipline makes the accidental mixed mode structurally impossible; (iv) state the freeze-or-not-fine-tune decision as a bias-variance trade-off in the running statistics, and choose correctly for small and large fine-tuning sets.

(b) Transfer probe. A learner’s training loop calls evaluate every 10 epochs, and evaluate opens with model.eval() and does not restore the mode. The first evaluate call is at epoch 10, and the training loss at epoch 11 is 0.05 while the training loss at epoch 9 was 0.04 — the loss increased at epoch 11, then continued decreasing from epoch 12. What happened? Answer sketch: the evaluate at epoch 10 left the model in eval mode, so epoch 11’s training ran in eval mode (dropout off, batch norm frozen), and the loss increased because the eval-mode function is a different function than the training-mode function — the optimizer’s state and the training data’s gradient are for the training-mode function, and the eval-mode function’s loss is higher. The training at epoch 12 is back in training mode (because train_step opens with model.train() in the function-boundary discipline), and the loss resumes its decrease from the epoch-11 value (which is slightly higher than epoch 9’s). The signature is a one-epoch spike at the first evaluate, and it is diagnostic of the missing restore.

(c) Remediation rule. If you cannot do (b), return to §L.4.2 and redo WE-L.4.d, then re-attempt (b) by asking what mode the training at epoch 11 ran in.

Part E — Failure Taxonomy, Lab, Cheat Sheet, Exercises, Exit Test

§L.5 — Failure taxonomy

The table covers the lab’s three break-it steps (batch-size-1, dropout tremor, stale-validation) plus three not exercised there (fine-tuning plateau, epoch-11 spike, resume).

§L.6 — The lab

§L.6.1 — Task

Build a BatchNorm1d + Dropout policy on an 8-step pose window, train it, and use it to reproduce, predict, and fix three mode bugs: the batch norm’s train-mode normalization of the window, the dropout tremor, and the stale validation.

§L.6.2 — Conditions

Python 3.10+, PyTorch 2.2+, CPU only. All runs use torch.manual_seed(42). The dataset is 6,000 eight-step windows generated from the arm’s 3-link planar reduction — the same injective map as Lesson 0.1.3’s lab, extended along a trajectory. The observation is (3, T) with T = 8: one channel per end-effector coordinate, over the window. The target is the joint configuration at the window’s end, (3,). The model is BatchNorm1d(3) → flatten → Linear(3T, 128) → ReLU → Linear(128, 128) → ReLU → Dropout(0.1) → Linear(128, 3). Training is 80 epochs at batch_size=64, Adam(lr=3e-3), CosineAnnealingLR(T_max=80), MSELoss(). Runtime: 6 seconds on CPU.

The observation is deliberately three-dimensional and windowed. A 2-D (B, 3) input is the shape at which PyTorch’s batch-size guard fires and the bug becomes loud; the (B, 3, T) window is the shape at which it does not, and is also what a real policy sees.

§L.6.3 — Controls held fixed

The initialization seed (0), the dataset, the architecture, the batch size (64), the learning rate (3e-3), the epoch count (80), and num_workers=0 are fixed. The only variable across runs is the mode configuration — and, in Step 9, the state of the batch-norm buffers, which the lab snapshots and restores so that each measurement starts from the trained model.

§L.6.4 — Metrics table

§L.6.5 — Pre-registered hypotheses

• H1. The correct pipeline’s val_loss is below 0.20 and below 1/10 of the mean-predictor baseline (1.870065). Prediction: a two-sided threshold, because a loss that merely beats the baseline has not shown the pipeline works.
• H2. The train-mode deployment of one window is at least 5× worse than the eval-mode deployment, and the train-mode BatchNorm1d output’s per-channel mean across T equals the layer’s trained β to four decimals, with per-channel variance γ² (not 1 — the layer’s trained γ is not 1). Prediction: a ratio > 5 and mean ≈ β; the second is the mechanism, and it is falsified if the mean is anything else.
• H3. The 20-forward action std is exactly 0.0 in eval mode and > 1e-3 in train mode. Prediction: two numbers, and the eval-mode 0.0 is exact, not approximate.
• H4. The train-mode validation loss exceeds the eval-mode one by a factor between 1.2 and 1.5; and running the buggy validation moves the running buffers, so the eval-mode validation measured afterwards differs from the one measured before. Prediction: two statements — a ratio in that band, and a nonzero buffer_drift. A learner who predicts only the first has missed the part that persists after the bug is fixed.

§L.6.6 — The break-it sequence

1. The batch-norm window collapse (silent at this shape). Deploy one (1, 3, T) window in training mode. Predicted symptom: no exception — the per-channel count is T = 8, so PyTorch’s guard passes — and the layer’s output has per-channel mean equal to β and variance γ², having discarded the window’s own mean. Actual: as predicted. Measured: input means [−0.058, −0.190, 5.087] become output means [0.158, 0.081, −0.149], which is β to four decimals; the deployment loss is 13.19× the eval-mode loss. The same layer on a (1, 3) input raises — the guard is real, and it is a per-channel count test, not a batch-size test.
2. The dropout tremor (silent). Forward the same window 20 times in training mode. Predicted symptom: no exception; the 20 actions differ; the std is > 1e-3. Actual: as predicted — 0.185750 in train mode against exactly 0.0 in eval mode.
3. The stale validation (silent, and it corrupts state). Compute the validation loss with model.train(). Predicted symptom: no exception; the reported loss is higher than the eval-mode loss; and because every train-mode forward pass updates the batch-norm running buffers in place, the model is changed by the act of measuring it. Actual: as predicted — the stale number is 1.302× the eval-mode number, and the buffers move by up to 0.0233 in running_mean and 0.0729 in running_var. The eval-mode validation measured after the bug reads 0.976× the one measured before.

All three are silent. The third is the one worth the extra step: a validation that reports a worse number is annoying, but a validation that modifies the model it is measuring is a correctness bug, and the two are one line apart.

§L.6.7 — Acceptance criteria

§L.6.8 — What good looks like

A strong submission has five properties:

1. The collapse is shown as mean = β, not as “much worse.” The learner prints the batch-norm layer’s input means and output means side by side and shows that the output mean is the trained bias, not the input’s mean. The number is the mechanism; the loss ratio is only its consequence.
2. The guard and the silence are shown together. The learner prints the ValueError from the (1, 3) input and the quiet, β-valued output from the (1, 3, T) input. A learner who reports only the raise has learned a rule that fails at their own deployment shape; a learner who reports only the silence has not found the boundary.
3. The tremor is shown as a std across 20 identical forwards, and the eval-mode value is reported as exactly 0.0 — not “small.”
4. The stale validation is measured for both of its effects. The learner reports the ratio and the buffer drift, and states that the second is the one that survives the fix: correcting the eval() call restores the number, but the buffers have already moved and only a snapshot or a retrain restores them.
5. Every measurement starts from the trained model. The learner snapshots state_dict() after training and restores it before each mode experiment, because every train-mode forward pass mutates the buffers. A notebook whose later experiments inherit the earlier ones’ buffer changes is measuring a different model each time.

§L.6.9 — Runnable lab code

# lab_0_1_4.py — mode-sensitive layers, the two-mode function, and the mode bugs
# Requires: Python 3.10+, PyTorch 2.2+, CPU only.

import copy
import math
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader

torch.manual_seed(42)
device = torch.device("cpu")
T = 8   # timesteps per observation window

# --- Step 1: the model, with both mode-sensitive layers ------------------
class IKNetMode(nn.Module):
    def __init__(self):
        super().__init__()
        self.bn = nn.BatchNorm1d(3)      # mode-sensitive: batch stats vs. running stats
        self.fc1 = nn.Linear(3 * T, 128) # mode-insensitive
        self.fc2 = nn.Linear(128, 128)   # mode-insensitive
        self.drop = nn.Dropout(0.1)      # mode-sensitive: mask vs. identity
        self.fc3 = nn.Linear(128, 3)     # mode-insensitive
    def forward(self, x):
        x = self.bn(x)                   # x is (B, 3, T): BN1d normalizes per channel
        x = x.flatten(1)
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = self.drop(x)
        return self.fc3(x)

model = IKNetMode().to(device)
print(f"bn.training = {model.bn.training}, drop.training = {model.drop.training}")
# Expected: both True. `training` defaults to True at construction.

# --- Step 2: the dataset — an 8-step pose window, predicting the final pose ---
# The observation is (3, T): one channel per end-effector coordinate, over the
# window. The target is the joint configuration at the window's end. The task
# is well posed: the 3-link planar reduction is injective (Lesson 0.1.3, L.6.10),
# so a window determines its endpoint.
def fk3(q):
    q1, q2, q3 = q[..., 0], q[..., 1], q[..., 2]
    return torch.stack([torch.cos(q1) + torch.cos(q1 + q2) + torch.cos(q1 + q2 + q3),
                        torch.sin(q1) + torch.sin(q1 + q2) + torch.sin(q1 + q2 + q3),
                        q1 + q2 + q3], dim=-1)

N = 6000
q0 = torch.stack([(torch.rand(N) * 2 - 1) * math.pi,
                  0.15 + (math.pi - 0.3) * torch.rand(N),
                  0.15 + (math.pi - 0.3) * torch.rand(N)], dim=-1)
dq = (torch.rand(N, 3) - 0.5) * 0.25                       # per-step joint velocity
steps = torch.arange(T + 1).float().view(1, T + 1, 1)
qs = q0[:, None, :] + dq[:, None, :] * steps               # (N, T+1, 3)
poses = fk3(qs)                                             # (N, T+1, 3)
perm = torch.randperm(N)
qs, poses = qs[perm], poses[perm]
X = poses[:, :T, :].transpose(1, 2)                         # (N, 3, T)
Y = qs[:, T, :]                                             # (N, 3)

class WindowDataset(Dataset):
    def __init__(self, x, y):
        self.x, self.y = x, y
    def __len__(self):
        return len(self.x)
    def __getitem__(self, i):
        return self.x[i].clone(), self.y[i].clone()

train_ds = WindowDataset(X[:4800], Y[:4800])
val_ds = WindowDataset(X[4800:], Y[4800:])
train_loader = DataLoader(train_ds, batch_size=64, shuffle=True, num_workers=0)
val_loader = DataLoader(val_ds, batch_size=256, shuffle=False, num_workers=0)
loss_fn = nn.MSELoss()

# --- Step 3: the function-boundary discipline ---------------------------
def train_step(model, optimizer, x, y):
    model.train()                       # set the mode at entry
    optimizer.zero_grad()
    loss = loss_fn(model(x), y)
    loss.backward()
    optimizer.step()
    return loss.item()

def evaluate(model, loader):
    model.eval()                        # set the mode at entry
    with torch.no_grad():
        total = sum(loss_fn(model(x), y).item() for x, y in loader) / len(loader)
    model.train()                       # restore for the caller
    return total

# --- Step 4: train, and measure the floor (H1) --------------------------
optimizer = torch.optim.Adam(model.parameters(), lr=3e-3)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=80)
for epoch in range(80):
    for x, y in train_loader:
        train_step(model, optimizer, x, y)
    scheduler.step()

val_loss = evaluate(model, val_loader)
baseline = ((val_ds.y - train_ds.y.mean(0)) ** 2).mean().item()
print(f"H1 val_loss (eval mode) = {val_loss:.6f}   mean-predictor baseline = {baseline:.6f}")

# A snapshot, because several steps below run forward passes in TRAIN mode,
# and every train-mode forward pass updates bn's running buffers in place.
trained_state = copy.deepcopy(model.state_dict())

x_single = val_ds[0][0].unsqueeze(0)     # (1, 3, T)
y_single = val_ds[0][1].unsqueeze(0)     # (1, 3)

def restore():
    model.load_state_dict(trained_state)
    return model

# --- Step 5: the guard — PyTorch refuses the total collapse (H2) --------
# The arithmetic says a batch of 1 gives (x - x) / sqrt(0 + eps) = 0. PyTorch
# never runs that arithmetic on a 2-D input: _verify_batch_size rejects any
# input whose per-channel element count is 1, BEFORE the statistics are formed.
model.train()
try:
    model.bn(torch.randn(1, 3))          # 2-D input, batch of 1
    print("2-D batch of 1 in train mode: NO RAISE")
except ValueError as e:
    print(f"2-D batch of 1 in train mode raises: {e}")

# --- Step 6: what the guard does not catch — the silent collapse --------
# Two IDENTICAL rows: per-channel count is 2, so the guard passes, the variance
# is exactly 0, and the output is exactly beta. The total collapse, no exception.
bn = nn.BatchNorm1d(3)
bn.train()
row = torch.tensor([1.0, 2.0, 3.0])
print(f"BN(2 identical rows) = {bn(torch.stack([row, row])).tolist()}")

# The same collapse, one dimension up: the model's own BN on a (1, 3, T) window.
# Per-channel count is T = 8, so the guard passes — and the layer removes the
# window's own mean, which is signal the network was meant to see.
model = restore()
model.train()
with torch.no_grad():
    bn_train = model.bn(x_single)
    bn_eval = model.bn(x_single)         # still train mode: same call, same result
model.eval()
with torch.no_grad():
    bn_eval = model.bn(x_single)
print(f"input, mean across T:                 {[round(v, 4) for v in x_single.mean(-1).flatten().tolist()]}")
print(f"train-mode BN output, mean across T:  {[round(v, 6) for v in bn_train.mean(-1).flatten().tolist()]}")
print(f"train-mode BN output, var  across T:  {[round(v, 6) for v in bn_train.var(-1, unbiased=False).flatten().tolist()]}")
print(f"the layer's trained bias (beta):      {[round(v, 4) for v in model.bn.bias.tolist()]}")
print(f"eval-mode BN output, mean across T:   {[round(v, 4) for v in bn_eval.mean(-1).flatten().tolist()]}")

# --- Step 7: the deployment cost of the collapse (H2, continued) --------
model = restore()
model.eval()
with torch.no_grad():
    loss_eval = loss_fn(model(x_single), y_single).item()
model = restore()
model.train()
with torch.no_grad():
    loss_train = loss_fn(model(x_single), y_single).item()
print(f"H2 deploy loss, eval mode = {loss_eval:.6f}   train mode = {loss_train:.6f}   "
      f"ratio = {loss_train / loss_eval:.2f}")

# --- Step 8: the dropout tremor (H3) -----------------------------------
model = restore()
with torch.no_grad():
    model.train()
    train_actions = torch.stack([model(x_single) for _ in range(20)])
    model.eval()
    eval_actions = torch.stack([model(x_single) for _ in range(20)])
print(f"H3 train-mode action std = {train_actions.std(0).mean().item():.6f}")
print(f"H3 eval-mode  action std = {eval_actions.std(0).mean().item():.6f}")

# --- Step 9: the stale validation, and the buffer it corrupts (H4) -----
# The bug: the validation loop forgets model.eval(). Two things go wrong at
# once — the reported number is a train-mode number, and the buffers move.
model = restore()
val_before = evaluate(model, val_loader)
rm_before = model.bn.state_dict()["running_mean"].clone()
rv_before = model.bn.state_dict()["running_var"].clone()

model.train()                            # the bug: no model.eval() here
with torch.no_grad():
    val_stale = sum(loss_fn(model(x), y).item() for x, y in val_loader) / len(val_loader)

val_after = evaluate(model, val_loader)
rm_after = model.bn.state_dict()["running_mean"]
rv_after = model.bn.state_dict()["running_var"]
print(f"H4 eval-mode val before the bug = {val_before:.6f}")
print(f"H4 train-mode val (the bug)     = {val_stale:.6f}   "
      f"ratio = {val_stale / val_before:.3f}")
print(f"H4 eval-mode val after the bug  = {val_after:.6f}   "
      f"ratio = {val_after / val_before:.3f}")
print(f"H4 running_mean moved by up to {float((rm_after - rm_before).abs().max()):.4f}")
print(f"H4 running_var  moved by up to {float((rv_after - rv_before).abs().max()):.4f}")

# --- Step 10: eval alone vs. eval + no_grad (M5) -----------------------
model = restore()
model.eval()
with torch.no_grad():
    out_no_grad = model(x_single)
out_grad = model(x_single)
print(f"M5 identical outputs: {torch.allclose(out_no_grad, out_grad, atol=1e-6)}")
print(f"M5 requires_grad: no_grad -> {out_no_grad.requires_grad}, "
      f"eval alone -> {out_grad.requires_grad}")

# --- Step 11: summary --------------------------------------------------
print("\n--- lab 0.1.4 summary ---")
print(f"H1 val_loss {val_loss:.6f} vs baseline {baseline:.6f}")
print(f"H2 deploy ratio (train / eval) = {loss_train / loss_eval:.2f}")
print(f"H3 train std {train_actions.std(0).mean().item():.6f} vs eval std "
      f"{eval_actions.std(0).mean().item():.6f}")
print(f"H4 stale ratio {val_stale / val_before:.3f}, "
      f"post-bug drift {val_after / val_before:.3f}")
print("silent break-its: #1 (BN removes the window mean), #2 (dropout tremor), "
      "#3 (stale validation)")

Run the file end to end. It takes about 6 seconds and prints:

bn.training = True, drop.training = True
H1 val_loss (eval mode) = 0.100740   mean-predictor baseline = 1.870065
2-D batch of 1 in train mode raises: Expected more than 1 value per channel when training, got input size torch.Size([1, 3])
BN(2 identical rows) = [[0.0, 0.0, 0.0], [0.0, 0.0, 0.0]]
input, mean across T:                 [-0.0576, -0.1897, 5.0873]
train-mode BN output, mean across T:  [0.158344, 0.081326, -0.148611]
train-mode BN output, var  across T:  [0.545919, 0.527961, 0.797075]
the layer's trained bias (beta):      [0.1583, 0.0813, -0.1486]
eval-mode BN output, mean across T:   [0.1293, -0.083, 0.5599]
H2 deploy loss, eval mode = 0.124629   train mode = 1.643512   ratio = 13.19
H3 train-mode action std = 0.185750
H3 eval-mode  action std = 0.000000
H4 eval-mode val before the bug = 0.100740
H4 train-mode val (the bug)     = 0.131205   ratio = 1.302
H4 eval-mode val after the bug  = 0.098309   ratio = 0.976
H4 running_mean moved by up to 0.0233
H4 running_var  moved by up to 0.0729
M5 identical outputs: True
M5 requires_grad: no_grad -> False, eval alone -> True

Three lines deserve a second look. The BN(2 identical rows) line is the total collapse, reached without an exception: per-channel count is 2, the guard passes, the variance is exactly 0, and the output is exactly β. The beta line next to the train-mode output means the layer discarded the input’s mean and emitted its bias — that is the mechanism, and the 13.19× ratio is only its consequence. And H4 eval-mode val after the bug is lower than before the bug, which looks like an improvement and is not: the buggy validation moved the running buffers toward the validation data’s statistics, so the model now normalizes by numbers that were fit to the set it is being scored on. Measuring the model changed it.

If any step does not match, the failure taxonomy in §L.5 names the likely causes.

§L.7 — The cheat sheet

§L.8 — Exercises with solutions

E 0.1.4.1 — The batch-norm trace (numerical). A batch-norm layer has γ = 2, β = 1, ε = 0, momentum = 0.1, and a batch of x = [1, 3]. The running statistics start at μ_running = 0, σ²_running = 1. Compute the training-mode output, the eval-mode output, and the updated running statistics.

S 0.1.4.1. Training: μ_c = (1+3)/2 = 2, σ_c² = [(1-2)² + (3-2)²]/2 = 1, \hat{x} = [(1-2)/1, (3-2)/1] = [-1, 1], y = 2·\hat{x} + 1 = [-1, 3]. Running: μ_running ← 0.9·0 + 0.1·2 = 0.2, and σ²_running ← 0.9·1 + 0.1·2 = 1.1 — the buffer update uses the unbiased batch variance 2 = 4/2, not the biased 1 that normalized the batch. Eval: \hat{x} = [(1-0.2)/sqrt(1.1), (3-0.2)/sqrt(1.1)] = [0.763, 2.670], y = 2·\hat{x} + 1 = [2.526, 6.339]. The lesson: the training-mode and eval-mode outputs are different functions of the same input; the difference is entirely in the statistics (2, 1 vs. 0.2, 1.1), and the affine transform (2, 1) is the same in both. It also matters which variance each step consumes: the normalization takes the biased one, the buffer update the unbiased one.

E 0.1.4.2 — The batch-size-1 collapse (numerical, with a shape twist). A batch-norm layer with γ = 1, β = 0, ε = 1e-5 receives a single example in training mode. What is the output for an input of shape (1, 3)? What is the output for an input of shape (1, 3, 4) whose per-channel values across the length axis are [1, 3, 3, 1] in channel 0? What is the output in eval mode with μ_running = 2, σ²_running = 4?

S 0.1.4.2. For (1, 3), the call raises ValueError: Expected more than 1 value per channel when training — the arithmetic (x − x)/sqrt(0 + ε) = 0 is never reached. For (1, 3, 4) with channel 0 equal to [1, 3, 3, 1], the per-channel count is 4, the guard passes, μ_c = 2 and σ_c² = 1, so the output is [(1−2), (3−2), (3−2), (1−2)]/sqrt(1 + 1e-5) ≈ [−1.000, 1.000, 1.000, −1.000] — the layer removed the window’s own mean of 2, which is signal. A degenerate window whose values are all equal would give σ_c² = 0 and output β = 0 throughout. In eval mode: \hat{x} = (x − 2)/sqrt(4 + 1e-5), a proper function of each input value. The lesson: “batch size 1” is not one behavior. The framework raises when the per-channel count is 1 and stays silent when it is greater, so the same conceptual bug is loud on a (B, C) input and silent on a (B, C, L) one.

E 0.1.4.3 — The dropout expectation (transfer probe). A Dropout(p = 0.25) layer receives x = [4, 8]. What is the expected output over the mask distribution? What is the variance of each element?

S 0.1.4.3. Each element is kept with probability 1-p = 0.75 and scaled by 1/(1-p) = 4/3. E[y_i] = 0.75 · (4/3) · x_i + 0.25 · 0 · x_i = 1.0 · x_i = x_i — the expectation is the input, [4, 8]. The variance is Var[y_i] = x_i² · p/(1-p) = x_i² · 0.25/0.75 = x_i²/3. For x_1 = 4: 16/3 ≈ 5.33; for x_2 = 8: 64/3 ≈ 21.33. The lesson: the inverted scaling makes the expectation exactly the input, so the eval mode’s identity is the unbiased estimate of the training-mode’s average; the variance is per-element and grows with p/(1-p), so p = 0.5 gives Var = x² (the largest for p < 1), and p = 0.9 gives Var = 9x² (larger still).

E 0.1.4.4 — The diagnostic exercise (diagnostic). A learner’s policy works at training batch size 64 and fails at deployment batch size 1. The training-time validation (batch size 64, model.eval()) shows 95% success, and the deployment (batch size 1, model.eval()) shows 15%. They have confirmed that the deployment’s model.eval() is called. What is the most likely cause?

S 0.1.4.4. The deployment is in eval mode, so the batch-size-1 collapse of training-mode batch norm does not apply; the remaining candidate is a different batch-size dependence: the model may have nn.BatchNorm1d in the backbone whose running statistics were computed at batch size 64, and the eval-mode batch norm’s output at batch size 1 uses those running statistics — which is correct and batch-independent. So the deployment should work. The failure with model.eval() confirmed is therefore not the batch-norm collapse; it is likely (1) the validation and the deployment use different observation pipelines (a normalization applied in the training that is not applied in the deployment, or vice versa), or (2) the deployment’s model.eval() is called on the top-level model but a submodule added later is still in training mode (a mixed-mode leak), or (3) the batch norm’s running statistics were not saved in the checkpoint, so the deployment’s running statistics are the initial values (0, 1), not the trained values. The discriminating test for (3) is to print model.bn.running_mean after loading the checkpoint and compare against the training’s value. The lesson: “eval mode is set” is necessary and not sufficient; the mode audit must include every submodule and the checkpoint’s running statistics, and the failure of a confirmed-eval deployment is a different bug than the mode bug, which is the point of the audit.

E 0.1.4.5 — The diagnostic exercise (diagnostic). A learner’s fine-tuning plateaus at a validation loss 3× worse than training from scratch on the same data. Their fine-tuning code is model.backbone.eval() throughout, with the head in training mode. What is the most likely cause, and what is the fix?

S 0.1.4.5. The model.backbone.eval() freezes the backbone’s batch norm running statistics at their pre-training values, which do not match the fine-tuning data’s distribution (a different camera, a different arm, a different workspace). The frozen statistics apply the pre-training standardization to the fine-tuning data, so the backbone’s features are miscalibrated, and the head’s training cannot compensate — the loss plateaus at a value determined by the miscalibration. The fix is to not freeze the batch norm (model.backbone.train() for the batch-norm layers, or remove the model.backbone.eval() and use model.train() for the whole model), if the fine-tuning set is large enough to estimate the running statistics (which, for a “same data” comparison against scratch, it is). The bias-variance trade-off of §L.4.2 applies: a small fine-tuning set would favor freezing, and a large one would favor adapting. The lesson: the model.backbone.eval() is a statistical choice, not a convention, and the choice must be matched to the fine-tuning set’s size; the learner who freezes “for safety” has chosen the high-bias side of the trade-off, and the plateau is the cost.

E 0.1.4.6 — The epoch-11 spike (diagnostic, transfer probe). A learner’s training loss is 0.04 at epoch 9, 0.05 at epoch 11, and 0.045 at epoch 12, with a validation at epoch 10. What is the most likely cause, and how would the learner confirm it in one experiment?

S 0.1.4.6. The cause is a mode leak from the epoch-10 validation: evaluate set model.eval() and did not restore model.train(), so the epoch-11 training ran in eval mode. The eval-mode function is different from the training-mode function (dropout off, batch norm frozen), and the optimizer’s state is for the training-mode function; the loss at epoch 11 is the eval-mode loss, which is higher by the dropout’s variance (and by the batch norm’s frozen statistics’ miscalibration). At epoch 12, the training is back in training mode (because train_step opens with model.train() in the function-boundary discipline, or because the train() was restored), and the loss resumes its decrease from the epoch-11 value. The confirming experiment is to print model.training at the start of each train_step; if it is False at epoch 11’s first step, the leak is confirmed. The lesson: the one-epoch spike is the signature of a mode leak, and the leak is detectable by logging the mode at each phase’s entry — the cheapest diagnostic, and the one that makes the function-boundary discipline’s absence visible.

E 0.1.4.7 — The MC dropout std (constructive, transfer probe). A learner takes 10 MC dropout samples and reports the per-sample std as the uncertainty. What is the relationship between their reported std and the correct posterior std, and what is the correction?

S 0.1.4.7. The per-sample std is σ (the dropout noise’s std), and the posterior std is σ/√n = σ/√10 ≈ 0.316·σ. The learner’s reported std is √n ≈ 3.16× larger than the correct posterior std. The correction is to divide the stacked samples’ std(0) by √n — torch.stack(samples).std(0) / math.sqrt(n) — because std(0) estimates the per-sample σ, not the std of the mean. The learner who uses the per-sample std has a safety threshold that is 3.16× too conservative, and the system requests human assistance far more often than necessary. The lesson: the MC dropout’s uncertainty is the std of the mean, and std(0) alone does not give it; the factor of √n is the difference, and the correction is the division by √n.

E 0.1.4.8 — The mode audit (constructive). A learner’s VLA has LayerNorm in the transformer, MultiheadAttention(dropout=0.1) in the transformer, Dropout(0.1) at the output, and BatchNorm1d in the image patch embedding. Write the mode audit: which layers are mode-sensitive, what state each uses, and what the deployment’s mode must be.

S 0.1.4.8. Mode-sensitive: MultiheadAttention(dropout=0.1) (its internal dropout is a mask in training, identity in eval), Dropout(0.1) at the output (mask vs. identity), and BatchNorm1d in the patch embedding (per-batch statistics vs. running statistics). Mode-insensitive: LayerNorm (feature-wise, no batch statistic). State: the MultiheadAttention’s dropout and the output Dropout have plain attributes (the rate) and no parameters or buffers; the BatchNorm1d has parameters (γ, β) and buffers (running_mean, running_var). Deployment’s mode: model.eval(), which sets all four to eval — the attention’s dropout off, the output’s dropout off, the batch norm on running statistics, and the LayerNorm unchanged. The lesson: the audit is a table, one row per layer, with three columns (mode-sensitive?, state kind, deployment’s mode); the table is the learner’s understanding made visible, and the absence of a row for a layer is the absence of the audit.

§L.9 — Exit test

You are ready for Lesson 0.2.1 when you can, without notes:

1. State the single mechanism by which train() and eval() change behavior, and the default mode of a freshly constructed module.
2. Classify a given architecture’s layers as mode-sensitive or mode-insensitive, and state which of the four kinds of module state each mode-sensitive layer’s state belongs to.
3. Trace a batch-norm forward pass in both modes by hand, including the running-statistics update, and state the output’s difference between the modes.
4. State the two batch-size failures — the ValueError at per-channel count 1 and the silent collapse to β at a degenerate batch — and say which shape triggers which.
5. State the training and eval functions of dropout, the inverted scaling’s purpose, and the expectation/variance relationship between them.
6. State the MC dropout pattern and the difference between the posterior’s std (σ/√n) and the per-sample’s std (σ).
7. Diagnose a “works at batch size 64, fails at batch size 1” deployment as a mode bug, and name the confirming test.
8. Diagnose a “trembling robot” as a dropout-in-training-mode bug, and name the confirming test.
9. Diagnose a “fine-tuning plateau” as a frozen-batch-norm bug, and state the bias-variance trade-off that determines whether to freeze.
10. Implement the function-boundary discipline, with train_step and evaluate each setting their own mode at entry, and state why it makes the accidental mixed mode structurally impossible.
11. State the difference between model.eval() and torch.no_grad() in terms of the function vs. the graph, and the symptom of calling only eval().
12. State the “one-epoch spike” as the signature of a mode leak, and name the cheapest diagnostic for it.

If any of items 1–12 is not answerable without notes, return to the section that introduced it: item 1 to §L.1, item 2 to §L.1.3, item 3 to §L.2.2, item 4 to §L.2.2, item 5 to §L.3.2, item 6 to §L.3.4, item 7 to §L.2.2 and §L.2.4, item 8 to §L.3.2, item 9 to §L.4.2, item 10 to §L.4.2, item 11 to §L.4.3, and item 12 to E 0.1.4.6.

§L.10 — Provenance and evidence tiers

What this lesson reproduces. Every hypothesis in §L.6.5 is executed by the lab in §L.6.9. Tier A, status R.

What this lesson does not reproduce. The Bayesian reading of MC dropout — that keeping dropout on at inference samples an approximate posterior over the network’s functions — is a theoretical claim from the variational-inference literature. It is tier B, status S: standard, and not tested here. The lesson measures what the interpretation requires: that the samples differ, that their mean is the eval-mode output, and that the std of the mean falls as σ/√n. Those three are tier A and are what the design principle actually rests on. The step from “the samples have the right statistics” to “they are posterior samples” is the literature’s.

One claim is deliberately weaker than it could be stated. The lesson says the training-mode and eval-mode batch-norm outputs “coincide only in the limit of an infinite batch from a stationary distribution; in practice they differ by an amount that depends on the batch size and the distribution.” The first clause is exact and is a limit statement, not a measurement. The second is a qualitative claim about a magnitude the lesson does not fit — tier C, status P — and it is phrased qualitatively for that reason. A reader who wants the magnitude should measure it; the lab’s H2 gives one instance of it (the 13.19 ratio) and that instance is not a law.

No external sources are cited. The lesson is about library behavior the reader runs, plus one interpretation marked above.