Lesson 0.1.1 — Tensors as the Substrate of Robot Learning: Shapes, Broadcasting, and Device Management

Week 1 · Prerequisites: Python fluency (no prior lesson ID; this is the first lesson of the curriculum).

Prior results used. None from the curriculum; the lesson uses only the learner’s Python fluency, and it restates at the point of use the small amount of array semantics it borrows from that background.

Hands forward. This lesson hands forward one capability: the learner can read a tensor expression, determine its shape, dtype, and device at every step, and predict which operations will silently produce the wrong shape rather than raising — the capability that Lesson 0.1.2 (autograd) and every subsequent lesson assumes.

Running case. A 6-DOF simulated robot arm in MuJoCo that emits, at each control timestep, a joint-angle observation vector of length 6 and receives a joint-velocity command vector of length 6; every example in this lesson is an instance of a tensor that holds one of these vectors, a batch of them across parallel environments, or a temporal stack of them across a rollout.

§L.0 — Lesson map, objectives, and misconceptions

§L.0.1 — Lesson map

§L.0.2 — Objectives

By the end of this lesson you will be able to do seven things:

1. State the five attributes that define a tensor — shape, dtype, device, requires_grad, strides — and, for any tensor expression, identify the value of each attribute at every intermediate step.
2. Trace a tensor expression through a concrete input, writing down the shape, dtype, and device at each operation, and predict the final shape before evaluating.
3. Distinguish a view from a copy: given two operations on a contiguous tensor, decide which returns a view, which returns a copy, and what the consequence is for in-place modification.
4. Apply the broadcasting rule to a pair of shapes, and decide in advance whether the broadcast is the one the programmer intended or a silent shape inflation.
5. Diagnose three classes of silent tensor bug — unintended broadcasting, device mismatch, and device retention in a log — from the symptom alone, without running the code.
6. Implement a tensor-sanity harness — shape assertions, device discipline, and metric extraction — that a training loop can use to fail loudly rather than silently.
7. Produce a written “shape audit” of a tensor pipeline: the sequence of shapes flowing through it, the operations that change rank, and the operations that could silently inflate rank.

§L.0.3 — Misconception inventory

The five misconceptions are not novice errors. Each is what a capable adult forms from partial exposure: a NumPy habit carried into PyTorch (M1, M3), a safety heuristic that happens to be unsound (M2), a true observation over-generalized (M4), and a performance instinct applied where the cost-benefit is inverted (M5). Each is displaced in the section named, and each displacement is tagged in the concept check that follows.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

This is the first lesson of the curriculum, so there are no prerequisite lessons from which to draw a spaced-retrieval warm-up. In place of a retrieval from prior lessons, the warm-up retrieves from the learner’s Python and NumPy background — the mental models the lesson will use as raw material — and probes them for the assumptions the lesson will revise. Answer each before reading on.

Warm-up 1. In NumPy, a = np.arange(12).reshape(3, 4) followed by a[0, 0] = 99 changes the value at index (0, 0) of a. Does np.reshape copy? Answer: No — reshape returns a view of the original buffer, so writing to the view writes to the original. How this is used: M3 in the misconception inventory is the PyTorch version of this same view semantics, and §L.2 makes the view/copy distinction precise for tensors.

Warm-up 2. In NumPy, a.shape == (3, 1) and b.shape == (3,); what is (a + b).shape? Answer: (3, 3) — NumPy and PyTorch both broadcast the trailing 1 against 3 and the leading 3 against an implicit 1, producing an outer-sum. How this is used: §L.3’s contrastive near-miss (WE-L.3.d) is exactly this case, framed as a robot-learning bug rather than a NumPy curiosity.

Warm-up 3. In Python, list1 + list2 concatenates; dict1 + dict2 raises. What does this tell you about operator semantics as a source of silent bugs? Answer: Operators are defined per type, and the same symbol can mean structurally different things; when two types are structurally similar, an operator that “works” may be doing the wrong structural thing. How this is used: broadcasting is the tensor case of this problem: + works on any broadcast-compatible pair, and “works” is not the same as “means what you meant.”

Warm-up 4. You are handed a training script and asked whether a tensor is on GPU. Without running it, what is the one expression in the script that determines the answer? Answer: The device= argument of the tensor’s creation, or the .to(device) call that moved it; every subsequent tensor inherits the device of its creation or of the last explicit move. How this is used: §L.4’s trace requires the learner to determine device at every step of a tensor pipeline from the creation and move calls alone.

Part A — What a Tensor Is

§L.1 — Objects: the five attributes

A tensor is a typed, multi-dimensional array that lives on a specific device and carries autograd metadata. The five attributes a learner must be able to name for any tensor expression are:

1. shape — a tuple of non-negative integers, one per dimension. The shape encodes the meaning of the axes: in a robot-learning pipeline, the first axis is conventionally the batch (or environment) axis, and the last axis is conventionally the feature axis. Intermediate axes are whatever the programmer’s data model says they are (time, channels, joints).
2. dtype — the numerical type of the elements. torch.float32 is the default for floating-point tensors and the type that most neural-network weights and activations use; torch.int64 is the default for integer tensors and the type that indices, class labels, and action tokens use; torch.bool is used for masks. Mismatches between dtypes (float32 activations against int64 targets) are a distinct class of bug from shape mismatches, and they raise more often than shape mismatches do — which is why they are the lesser concern for the learner of this lesson.
3. device — where the tensor’s memory is allocated. The two common values are cpu and cuda (the latter with an optional index for multi-GPU systems). An operation between tensors on different devices raises a RuntimeError; an operation on a CPU tensor in a GPU-bound training loop is silent — it runs, on the CPU, at a fraction of the throughput, and the learner wonders why the loop is slow.
4. requires_grad — whether autograd tracks operations on this tensor. This attribute is the subject of Lesson 0.1.2, and the present lesson only names it so that the learner can read it without being surprised.
5. strides — a tuple of integers, one per dimension, giving the number of elements to skip in memory to move one step along that dimension. For a contiguous (4, 6) float32 tensor, strides are (6, 1): moving one row skips six elements, moving one column skips one. Strides are the mechanism behind views, and they are the attribute that determines whether an apparently-new tensor shares memory with its source.

The formal tuple for a tensor is (data_buffer, shape, strides, dtype, device, requires_grad), where data_buffer is the underlying allocation and the remaining five are metadata describing how to interpret it. Two tensors with different shape and strides can share the same data_buffer; this is exactly what a view is, and it is why modifying one can modify the other.

§L.1.1 — Result: a tensor is defined by five attributes, and every operation preserves or changes each of them in a rule-governed way

Claim. For every operation f in the tensor algebra, the five attributes of f(x) are determined by the five attributes of x (and of f’s parameters, where it has them), and no operation changes an attribute without a rule that the programmer can state in advance.

Proof sketch. The attribute rules are:

• Shape is determined by the operation’s shape function: element-wise operations preserve shape; reductions remove axes; matrix multiplication combines the trailing axes of one operand with the leading non-batch axes of the other; indexing removes or reorders axes according to the index expression.
• dtype is determined by the operation’s promotion rule: float32 + float32 = float32; float32 + int64 = float32; int64 + int64 = int64; some operations force a dtype (torch.argmax returns int64 regardless of input).
• Device is determined by the operation’s placement rule: the output is on the device of the operands, and operands on different devices raise.
• requires_grad is True iff at least one operand has requires_grad=True and the operation is differentiable and not executed under torch.no_grad(); this rule is deferred to Lesson 0.1.2.
• Strides are determined by the operation’s memory layout rule: some operations return a contiguous output with default strides (arithmetic, reductions), some return a view whose strides encode a reinterpretation of the source buffer (reshape, permute, transpose, slicing), and some return a buffer that is neither contiguous nor a view (expand returns a view with zero strides, which is the trick that makes broadcasting memory-free).

The five rules are all of the same form — an operation is a function on the attribute tuple — and the learner who has internalized the form can, in principle, derive the attributes of any expression without running it. In practice, the learner needs to be fluent in the rules for the operations that appear most often in robot-learning pipelines, and the worked-example ladder below drills exactly those.

Observation, not result. The convention that the first axis is the batch axis is a convention, not a theorem. It is followed by PyTorch’s built-in layers and by essentially all robot-learning codebases (PyTorch’s “Learn the Basics” tutorial states the convention explicitly), but a tensor of shape (6,) is not intrinsically a batch of size 6; it is a vector that the programmer has chosen to treat as a batch. Misreading the convention is a source of bugs (a (6,) observation treated as a (1, 6) batch is a different tensor to a linear layer), and the learner must be able to state the convention and to identify when code violates it.

§L.1.2 — Worked-example ladder

WE-L.1.a — Fully worked: constructing and reading a tensor

Input. The running case: a 6-DOF arm in MuJoCo emits one observation vector per control timestep, containing the six joint angles. We create a single-step observation as a tensor of shape (6,).

Trace.

1. obs = zeros(6) — creates a contiguous float32 tensor on the default device (cpu if no other is specified). Attributes: shape (6,), dtype float32, device cpu, requires_grad False, strides (1,).
2. obs.fill_(0.1) — in-place fill; shape, dtype, device, strides unchanged; the values are now [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]. (The trailing underscore is part of the name: fill alone does not exist, and the call raises AttributeError.)
3. obs_2d = obs.reshape(1, 6) — returns a view; shape becomes (1, 6), strides become (6, 1), the data buffer is shared with obs, the other attributes are inherited.
4. obs_2d[0, 0] = 0.5 — in-place write to the view; because the view shares the buffer, obs[0] is now 0.5 as well.
5. obs.requires_grad_(True) — flips requires_grad to True; the other four attributes unchanged. The order matters: doing this before step 4 makes step 4 raise, a view of a leaf Variable that requires grad is being used in an in-place operation, because autograd cannot let a view of a leaf be overwritten silently. That raise is a loud failure and is the subject of Lesson 0.1.2.
6. obs_copy = obs.clone() — returns a copy; shape, dtype, device, strides preserved; new data buffer, independent of obs.

Output and interpretation. After step 4, obs and obs_2d are two views of the same six floats, and modifying one modifies the other. After step 6, obs_copy is an independent tensor. The single most common error in this trace is to assume step 3 copies — it does not — and the consequence is that a downstream modification to the reshaped tensor silently corrupts the original. This is M3 of the misconception inventory, and its displacement is the trace above.

Self-explanation 1. Before reading on: in step 3, why is reshape able to return a view rather than a copy? What property of obs makes this possible?

Answer. Because the target layout (1, 6) is reachable from obs’s layout by rewriting strides alone — obs is contiguous with strides (1,), and a (1, 6) view of the same six floats needs strides (6, 1), which is a legal reinterpretation of the same buffer. The general condition is stride reachability, not contiguity: torch.randn(4, 6)[:, ::2] is non-contiguous (strides (6, 2)) and still reshapes as a view, because (6, 2) reads as (12,) with stride 2. The case where reshape copies is one where no stride tuple can express the target order — a transposed tensor with no unit axis, or an expanded tensor.

Self-explanation 2. Before reading on: in step 5, what is the value of obs after obs_2d[0, 0] = 0.5? Why?

Answer. obs[0] is 0.5 and obs[1:] are unchanged, because the two tensors share the same data_buffer and the write goes to that buffer. The learner who expected obs to be unchanged has assumed a copy semantics that reshape does not provide.

WE-L.1.b — Completion problem

Input. Same running case. A batch of 32 parallel environments, each emitting a 6-dimensional observation, stacked into a single tensor of shape (32, 6).

Trace (with the final 25% replaced by prompts).

1. batch = zeros(32, 6) — attributes: shape (32, 6), dtype float32, device cpu, requires_grad False.
2. batch.stride() — ⟨your turn: what does this return, and why?⟩
3. batch_view = batch.view(32, 2, 3) — ⟨your turn: is this a view or a copy, and what are the strides?⟩
4. batch_view[0, 0, 0] = 7.0 — ⟨your turn: which element of batch is now 7.0?⟩

Answers.

1. Strides are (6, 1) because the tensor is contiguous: moving one row skips six elements, moving one column skips one.
2. A view, with strides (6, 3, 1). The buffer is unchanged; the shape has been reinterpreted as (32, 2, 3), and the strides are derived from the same (6, 1) base by re-splitting the trailing 6 into (2, 3).
3. batch[0, 0] is now 7.0, because the first element of the view is the first element of the buffer. The learner who wrote batch[0, 0, 0] as if batch were three-dimensional has misread the view; the correct answer is stated in the source tensor’s own shape.

The lesson. A view does not change the data buffer; it changes the interpretation of the buffer, and the interpretation is expressed in the strides. To know which element of the source a write to the view modifies, the learner computes the flat index $sum(i_k$ * stride_k) over the view’s strides and looks it up in the source’s own coordinate system.

WE-L.1.c — Faded: the running case with one parameter changed

Setup. The 6-DOF arm is now simulated across 8 parallel environments for a 50-timestep rollout. Each environment emits one 6-dimensional observation per timestep. The observations are stacked into a tensor of shape (8, 50, 6) — the convention here is (num_envs, rollout_length, obs_dim).

Question. The environment index is the first axis, the timestep index is the second, and the joint index is the third. What are the strides of this tensor, assuming it was created contiguously in this layout? What is the flat-memory offset of element (3, 17, 4)?

You must produce the intermediate steps. The setup and the answer are given; the intermediate computation is the learner’s.

Answer. The strides are (50 * 6, 6, 1) = (300, 6, 1). The flat offset of (3, 17, 4) is 3 * 300 + 17 * 6 + 4 * 1 = 900 + 102 + 4 = 1006. The learner who computed 3 * 6 + 17 * 1 + 4 * 50 has used the wrong stride order — the stride tuple follows the shape’s axis order, not the natural reading order of the index expression.

The lesson. The stride tuple is the mapping from logical indices to physical memory, and the mapping is offset = Σ $i_k$ · stride_k. Any question about which element a view modifies, or whether a reshape can be a view, reduces to this equation.

WE-L.1.d — Contrastive near-miss: tensor vs. as_tensor

Correct solution. as_tensor(x) converts a Python list to a tensor where possible without copying — if x is already a tensor, as_tensor(x) returns x (same buffer, same attributes); if x is a NumPy array, as_tensor(x) shares its buffer.

Near-miss. tensor(x) always copies — it constructs a new tensor from the data, regardless of what x was.

Why the near-miss produces a plausible but incorrect number. A learner who uses tensor(x) where as_tensor(x) was intended will not see a shape, dtype, device, or value error. The values will be numerically identical at the moment of construction. The bug only manifests later, when the source x is mutated (for example, by an in-place operation in a simulator callback, or by a data-augmentation pipeline) and the learner expects the tensor to reflect the mutation — it does not, because it is a copy. Or, in the reverse direction, the learner expects independence and uses as_tensor, and an in-place modification to the tensor corrupts the source. Either way, the failure is silent at construction and observed only at the point of mutation, often hundreds of lines downstream, where the connection between cause and symptom is not visible.

The discrepancy, stated. tensor and as_tensor differ in exactly one attribute of the result — whether the data buffer is shared with the input — and that attribute is not visible in any of the other four (shape, dtype, device, requires_grad are identical). The learner who inspects only the printed tensor sees no difference. The learner who understands that the buffer is the sixth attribute — the one the tensor’s repr does not show — knows to ask which of the two functions the code called.

Self-explanation 3. Before reading on: a colleague writes x = as_tensor(numpy_array) and then x[0] = 0. Is the NumPy array modified? What if they had written x = tensor(numpy_array)?

Answer. With as_tensor, yes — the tensor shares the array’s buffer, so the in-place write propagates. With tensor, no — the tensor owns a copy, and the array is unchanged. The choice between the two functions is a choice about ownership, and the choice is invisible unless the learner knows to look for it.

§L.1.3 — Comparative table: tensor construction

Design principle (P1.1). Allocate on the target device, not on CPU and then move. zeros(shape, device=target) allocates directly in the target memory; zeros(shape).to(target) allocates on CPU, then copies. In a training loop the difference is negligible; in a 50 Hz control loop that calls the allocation every timestep, the copy is measurable, and in a loop that runs on an embedded device with no CPU-GPU interconnect, the copy is prohibitive.

§L.1.4 — Concept check

Concept check (displaces M1). A learner says: “torch.tensor is the same as numpy.array, so I already know how to use it.” Which of the five attributes of a tensor has the learner failed to model, and what class of bug will that omission produce?

Answer. The learner has failed to model strides and the view/copy distinction that strides determine: numpy.array also has strides, but the NumPy habit treats reshape and slicing as value operations rather than views, and PyTorch exposes the sharing directly. The bug class is silent aliasing: a modification to a reshaped or sliced tensor corrupts the source, and a modification to the source corrupts the view, without an exception. The learner who says “same as NumPy” has also failed to model device, but the aliasing bug is the one that produces wrong numbers rather than a raised error.

§L.1.5 — Consolidation checkpoint 1

(a) You should now be able to: (i) name the five attributes of a tensor and state what each is determined by; (ii) trace a construction-and-modification sequence and state, at each step, whether a new buffer was allocated or an existing one was reinterpreted; (iii) decide whether a given function (tensor, as_tensor, from_numpy) copies or shares.

(b) Transfer probe. A colleague hands you a tensor y = x.permute(1, 0) where x has shape (4, 6). Without running the code, state the shape of y, its strides, and whether modifying y[2, 1] modifies x — and state which element of x it modifies. Answer sketch: y has shape (6, 4) and strides (1, 6) — the permute reorders the strides without touching the buffer — so modifying y[2, 1] modifies x[1, 2]. The learner who produced (6, 4) but could not state the strides has not yet internalized that the shape and the strides are independent attributes.

(c) Remediation rule. If you cannot do (a)(ii) or (b), return to §L.1.2 and redo WE-L.1.b, then re-attempt (b) without notes.

Part B — Shape Semantics

§L.2 — Objects: batch and feature axes, rank, views and copies

Rank is the number of dimensions, equal to the length of the shape tuple. A rank-1 tensor is a vector, rank-2 a matrix, rank-3 a stack of matrices, and so on. In robot learning, the rank is not arbitrary: it encodes the data model, and the data model is what the layer reads.

The batch axis is conventionally the first axis (axis 0). PyTorch’s nn.Linear expects (batch, in_features); nn.Conv2d expects (batch, channels, height, width); nn.RNN expects (batch, seq_len, features) by default. A tensor that violates the convention — a (6,) vector passed to a layer that expects (batch, 6) — will either raise (if the layer’s in-features do not match) or, worse, silently treat the 6 as the batch size and the nonexistent feature axis as the feature axis, producing a degenerate output. The single most common shape bug in robot learning code is a missing batch axis: a single observation of shape (6,) fed to a policy that was trained on (32, 6) and is now being evaluated at batch size 1.

The feature axis is conventionally the last axis. Intermediate axes, when present, carry whatever the programmer’s data model requires: seq_len for recurrent policies, num_joints for a joint-space action, channels for a visual observation, num_objects for a variable-length scene graph. The learner must be able to state, for any tensor in a pipeline, what each axis means — and must be able to do so before the tensor is passed to a layer whose shape expectations are positional.

A view is a tensor that shares its data buffer with another tensor and differs only in shape, strides, and possibly offset. A copy is a tensor with its own data buffer. The distinction matters because:

• A view is O(1) to create; a copy is O(N) in the number of elements.
• A write to a view is a write to the source; a write to a copy is not.
• A view’s lifetime keeps the source’s buffer alive; a copy’s does not.

The operations that return views, in the common case, are: reshape (when the target layout is reachable by stride rewriting, which every contiguous tensor satisfies), view (which requires contiguity and raises otherwise), permute/transpose, slicing x[a:b], expand, and x[i] (indexing with a single integer removes the axis but returns a view of the remaining sub-tensor, not a copy of the row). The operations that always return copies are: clone, arithmetic operations (the result is a new buffer), reductions (sum, mean), and contiguous (which returns the input unchanged if it is already contiguous, and a new contiguous copy otherwise).

§L.2.1 — Result: reshape returns a view iff the target layout is reachable by stride rewriting

Claim. For a tensor x, x.reshape(new_shape) returns a view of x when the element order implied by x’s existing strides coincides with the row-major order of the requested shape; otherwise it returns a copy. x.view(new_shape) returns a view when the same condition holds and raises a RuntimeError otherwise.

Proof sketch. A reshape is a view exactly when a stride tuple for the new shape reproduces the same flat-offset mapping. For shape ($e_0$, …, $e_m$) the only candidate strides are the row-major products ($e_1$⋯$e_m$, …, 1), so the question is whether x’s elements, traversed in index order, appear in memory at consecutive offsets 0, 1, 2, …. Contiguity is a sufficient condition — a contiguous tensor’s elements are already in that order by definition — but it is not necessary. A strided slice whose selected elements happen to be consecutive is also reachable: torch.randn(4, 6)[:, ::2] has stride (6, 2) and is_contiguous() is False, yet the three selected elements per row sit at offsets 0, 2, 4, and a reshape to (12,) therefore shares storage (verified). Conversely, x.t() on a (4, 6) input has stride (1, 6) and no stride tuple reproduces the row-major order of the transpose, so reshape copies (verified: data_ptr differs).

The practical consequence: reshape is safe in the sense that it always succeeds, but its view-versus-copy behavior depends on the element ordering of the input, which depends on the whole chain of operations that produced it — not on the single boolean is_contiguous() reports. view is strict: it refuses to produce a copy, so a learner who wants view semantics, and wants to be told when they cannot have them, should use view and catch the error.

§L.2.2 — Worked-example ladder

WE-L.2.a — Fully worked: the missing batch axis

Input. A trained policy expects observations of shape (batch, 6) and returns actions of shape (batch, 6). At deployment, a single observation from the simulator is a tensor obs of shape (6,).

Trace.

1. obs = zeros(6) — shape (6,), the observation vector.
2. pred = policy(obs) — the policy’s first layer is nn.Linear(6, 64), and a rank-1 input is accepted: PyTorch treats (6,) as a single sample with no batch axis, and the layer returns (64,) (verified). No exception is raised. This is the dangerous case, not the good one.
3. pred = policy(obs.unsqueeze(0)) — obs.unsqueeze(0) has shape (1, 6), the correct batch-of-1 form; pred has shape (1, 64). Every downstream operation that assumes a batch axis now works: batched normalization, batched dropout masks, and any code that reads pred.shape[0] as the batch size.
4. action = pred.squeeze(0) — shape (6,), the correct form for a simulator command.

Output and interpretation. The fix is the pair unsqueeze(0) (to add a batch axis for the policy) and squeeze(0) (to remove it for the simulator). Both are representational, not computational: no values change, only shape metadata. The bug this trace prevents is subtler than a crash. Because nn.Linear silently accepts rank-1 input, the missing batch axis survives the forward pass; what breaks is everything downstream that reads the rank. The downstream behavior splits three ways, and only one of the three is loud (all verified):

• Raises, but late and at the wrong layer. nn.BatchNorm1d rejects a 1-D input with ValueError: expected 2D or 3D input (got 1D input). The message names batch norm, not the missing unsqueeze, so the learner’s first instinct is to check the normalization layer’s configuration.
• Silent, with a plausible shape. pred.mean(0) on a (64,) tensor returns a scalar; on a (1, 64) tensor it returns a (64,) vector. torch.cat([pred, pred], dim=0) gives (128,) where the batched version gives (2, 64). Both are legal, and both are the shape the code will carry forward.
• Silent, with a warning that is easy to miss. F.mse_loss((6,), (1, 6)) returns a scalar after broadcasting and emits a UserWarning about mismatched target size — one line of stderr in a training loop that prints thousands.

nn.Dropout and nn.LayerNorm are indifferent to the batch axis, so they hide the missing axis entirely. The failure surfaces at the second step of a rollout, not the first, which is why the deployment symptom is “the policy worked once and then went wrong” rather than “the policy crashed.”

⟨Self-explanation 4. Before reading on: since nn.Linear(6, 64) accepts (6,) and returns (64,), what observable difference does the missing batch axis make at step 2 — and where does that difference first become visible?⟩

Answer. Nothing differs in the output values: policy(obs) and policy(obs.unsqueeze(0)).squeeze(0) are numerically identical (verified), because a linear layer applies the same matrix to one row either way. The difference is in the rank of the metadata the next operation reads, and the operations that read rank split into three classes: those that raise on a rank-1 input (BatchNorm1d), those that return a different shape without complaint (any reduction with dim=0, torch.cat along axis 0), and those that do not care (Dropout, LayerNorm, elementwise arithmetic). The missing axis is therefore invisible through the third class, loud at the first, and silently wrong at the second — which is the class a rollout loop hits first, because stacking per-timestep outputs along axis 0 is the most common thing a rollout loop does.

WE-L.2.b — Completion problem

Input. A batch of 32 observations of shape (32, 6) and a policy that internally reshapes to (32, 2, 3) for a structured action head.

Trace (final 25% as prompts).

1. batch = zeros(32, 6).
2. batch_3d = batch.view(32, 2, 3) — shape (32, 2, 3), strides (6, 3, 1), a view.
3. batch_3d_flat = batch_3d.reshape(32, 6) — ⟨your turn: is this a view or a copy? What are the strides?⟩
4. batch_3d[0, 1, 2] = 9.0 — ⟨your turn: which element of batch_3d_flat is now 9.0?⟩

Answers.

1. A view, with strides (6, 1) — the shape (32, 2, 3) is contiguous, so reshape back to (32, 6) is well-defined on the same buffer.
2. Element batch_3d_flat[0, 5] is now 9.0, because the flat offset of (0, 1, 2) in the (32, 2, 3) view is 0 * 6 + 1 * 3 + 2 * 1 = 5, and 5 in the (32, 6) view is index [0, 5].

The lesson. A reshape does not “materialize” a tensor; it reinterprets the buffer, and the reinterpretation is consistent across every view of the same buffer. A write anywhere in the view graph is visible everywhere.

WE-L.2.c — Faded: the running case with one parameter changed

Setup. The 6-DOF arm’s observation is a (8, 6) tensor across 8 environments, but the policy expects a (8, 1, 6) tensor to be consumed by a sequence-aware head. The learner uses obs.unsqueeze(1) to insert the middle axis.

Question. What are the shape and strides of obs.unsqueeze(1)? Is it a view? If a downstream layer returns a (8, 1, 6) tensor out and the learner does out.squeeze(1), what are the shape and strides of the result?

You must produce the intermediate steps. The setup and the answer are given; the intermediate computation is the learner’s.

Answer. obs.unsqueeze(1) has shape (8, 1, 6) and strides (6, 6, 1) — the inserted axis has a stride equal to the stride of the axis to its left, because a size-1 axis has no elements to skip. It is a view. out.squeeze(1) has shape (8, 6) and strides (6, 1) — it is the inverse of the unsqueeze, and it is a view. The learner who wrote (8, 1, 6) with strides (6, 1, 1) has confused the size-1 axis (which contributes nothing to the offset) with a rank-1 axis (which would contribute one element per step); the correct stride for the inserted axis is the left neighbor’s stride, not 1.

The lesson. unsqueeze inserts a size-1 axis and inherits the left neighbor’s stride; squeeze removes a size-1 axis and is the inverse. Both are views, both are O(1), and neither allocates.

WE-L.2.d — Contrastive near-miss: view after permute

Correct solution. The learner wants to flatten the first two axes of a (4, 6) tensor x to (24,). They write x.reshape(24) — which succeeds, because x is contiguous — and the result shares the buffer. Correct.

Near-miss. A second learner writes y = x.permute(1, 0) (shape (6, 4)) and then y.view(24). The result raises RuntimeError: view size is not compatible with input tensor's size and stride. The learner, surprised, tries y.reshape(24) instead, and it succeeds — but the result is a copy in a different memory order than x, and downstream arithmetic that assumed the flat order of x produces different results.

Why the near-miss produces a plausible but incorrect number. The reshape succeeds silently, and the resulting (24,) tensor has the correct values for the permuted order — but the permuted order is not the order the learner wanted. If the learner intended the flat order of x (row-major over x’s original shape), y.reshape(24) gives the column-major order of x instead, because y is the transpose. A loss computed on the reshaped tensor will be numerically different from the loss computed on the original, by an amount that depends on the data; a downstream linear layer will weight the elements in the wrong positions, and the model will train to a plausible-but-wrong solution.

The discrepancy, stated. view raises on a layout it cannot reinterpret, which is the learner’s friend: it tells them a copy would be required. reshape falls back to a copy in exactly those cases, which is the learner’s hazard: it succeeds, and the success is silent. The two functions diverge exactly when the learner has just applied permute or transpose with no unit axis — which is, in a vision pipeline, essentially always. Note that neither function’s behavior is a function of is_contiguous() alone: torch.randn(4, 6)[:, ::2] is non-contiguous and view still refuses it, yet reshape returns a view, because stride (6, 2) is a legal (12,) with stride 2.

Self-explanation 5. Before reading on: in the near-miss, the learner could have called y.contiguous().view(24). What does contiguous() do, and why does the combination succeed where view alone fails?

Answer. contiguous() returns the input unchanged if it is already contiguous, and a new contiguous copy of it otherwise — a copy whose buffer is laid out in row-major order for y’s shape. After contiguous(), the tensor is contiguous, so view(24) can reinterpret the (new) buffer with new strides without a further copy. The combination is the explicit way to say “I know this may copy, and I want a contiguous flat view of the permuted tensor”; the learner who uses reshape instead has written the same thing without the explicitness, and without the signal that a copy may have been made.

§L.2.3 — Comparative table: reshape-family operations

Design principle (P1.2). When you have just permuted or transposed, use view first and let it raise. The raise is information: it tells you the tensor you are about to flatten is not in the order you may have assumed, and it forces you to decide whether the order is correct before a silent copy bakes in the wrong assumption. reshape removes that signal.

§L.2.4 — Concept check

Concept check (displaces M3). A learner says: “reshape returns a new tensor, so I can modify the reshaped tensor safely.” Name the operation that would return an independent tensor, state the condition under which reshape returns a view, and name one pipeline in which that condition holds.

Answer. The operation that returns an independent tensor is clone(). reshape returns a view when the target layout is reachable from the source’s strides — which holds for every contiguous tensor, the default state of almost everything produced by arithmetic, reductions, or construction, so a (32, 6) tensor created by zeros(32, 6) or by a linear layer’s output reshapes as a view. The condition fails when no stride tuple can express the target order: after a permute/transpose with no unit axis, or on an expand with zero strides. It does not fail merely because is_contiguous() is False — a strided slice still reshapes as a view.

§L.2.5 — Consolidation checkpoint 2

(a) You should now be able to: (i) state the batch-first convention and name the class of bug it prevents (missing batch axis at deployment); (ii) decide, for any reshape/view/permute, whether the result is a view or a copy, and state the strides; (iii) trace the unsqueeze/squeeze pair and state the shape and strides of each intermediate.

(b) Transfer probe. A recurrent policy takes observations of shape (batch, seq_len, obs_dim) = (4, 32, 6). The learner wants to apply a per-timestep linear layer to the last axis only. They flatten to (4*32, 6), apply nn.Linear(6, 16), and reshape back to (4, 32, 16). Does the flatten-reshape round trip preserve the mapping between original (b, t) pairs and the reshaped (b, t) pairs? Answer sketch: Yes, because the flatten and reshape are both over the leading two axes in row-major order, and (4, 32, 6) is contiguous, so the round trip is the identity on the index pairs. The learner who is unsure should compute the flat offset of (b, t, j) in the original, the flat offset of (b*32 + t, j) in the flat tensor, and verify they are equal.

(c) Remediation rule. If you cannot do (b), return to §L.2.2 and redo WE-L.2.b, then re-attempt (b) with a pencil and paper for the offset computation.

Part C — Broadcasting

§L.3 — Objects: the broadcasting rule

Broadcasting is the operation by which two tensors of different shapes are combined element-wise by implicitly expanding the smaller one along dimensions of size 1. The rule is stated from the right:

Align the two shape tuples from the right. For each dimension position, the two sizes must be equal, or one of them must be 1 (in which case it is expanded to the other size), or one of the tuples must have run out of dimensions (in which case the missing dimensions are treated as size 1). If none of these holds, the operation raises.

The two common cases:

• Bias addition. A bias of shape (A,) added to a batch of logits of shape (B, A) is treated as (1, A) and expanded across the B axis. Result shape (B, A). Intended.
• Outer product. A column of shape (B, 1) added to a row of shape (B,) — which aligns to (1, B) — produces a (B, B) matrix. The learner who intended element-wise addition of a per-sample scalar to a per-sample column has instead computed the outer sum. Unintended, and silent.

The PyTorch tutorials name broadcasting explicitly as “the #1 source of silent bugs” in PyTorch code, and the reason is structural: broadcasting is designed to accept pairs of shapes that the programmer may not have meant, because the alternative — refusing every shape mismatch — would require the programmer to write .unsqueeze and .expand calls by hand, and those calls are the exact places where the learner would have to state their intention. Broadcasting is convenient precisely because it removes the places where the learner states intent. The price of that convenience is that an unintended broadcast is not a bug the framework can catch.

§L.3.1 — Result: shape compatibility is weaker than shape intention

Claim. There exist pairs of shapes (s, t) such that s and t are broadcast-compatible, the broadcast result is a well-defined tensor, and the result is not the tensor the programmer intended. Consequently, a passing tensor expression is not evidence of a correct tensor expression.

Proof sketch. The claim is witnessed by any pair of the form (B, 1) and (B,): they broadcast to (B, B), and the learner who intended (B, 1) has produced a silent shape inflation. The witness is not exotic — it is the shape of a per-sample scalar and a per-sample column, which is exactly the shape pair of a per-environment reward and a per-environment advantage in a policy-gradient update, and exactly the shape pair of a per-batch loss weight and a per-batch loss. The literature on PyTorch pitfalls catalogs broadcasting as the leading cause of silent shape bugs (the Berkeley deep RL course’s framing, tier C, P), and the reader can reproduce the witness in any PyTorch session.

The consequence for the learner is a procedural rule rather than a mathematical one: since the framework cannot distinguish intended from unintended broadcasts, the learner must introduce the distinction themselves, by asserting the shape of every tensor whose shape is not trivially determined by the expression. The assertion is the place where intention is stated, and it is the only such place.

Observation, not result. The choice of whether to assert on every tensor or only on those whose shape is non-obvious is a judgment, and reasonable engineers differ. In a training loop, asserting on the loss and the logits is usually sufficient; in a control loop, asserting on the action before it is sent to the actuator is the minimum, because the actuator is the place where a wrong-shape command becomes a physical event.

§L.3.2 — Worked-example ladder

WE-L.3.a — Fully worked: the intended broadcast

Input. A batch of 32 environments, each producing a scalar reward. The rewards are a (32,) tensor r. A discount factor gamma = 0.99 is a Python float. The learner computes the discounted return r * gamma.

Trace.

1. r = zeros(32) — shape (32,), dtype float32, device cpu.
2. gamma = 0.99 — a Python float, not a tensor. When used in an arithmetic expression with a tensor, PyTorch wraps it as a rank-0 tensor of the same dtype and device as the other operand (a scalar promotion, not a broadcasting operation).
3. discounted = r * gamma — shape (32,), dtype float32, device cpu. The multiplication is element-wise against the scalar.

Output and interpretation. The result has the same shape as r, because scalar multiplication does not broadcast in the sense that matters here — it is a special case in which the other operand is rank-0 and carries no shape to align. The learner should note that the trace here is trivial, and the point of including it is to establish the baseline against which the next trace is contrasted: the intended broadcast is the one in which the result’s shape is the same as the input’s, and the learner who sees a result shape different from the input’s should stop and ask whether the difference is intended.

Self-explanation 6. Before reading on: in step 2, why does PyTorch wrap the Python float rather than requiring the learner to call tensor(0.99)? What would be the cost of the requirement?

Answer. PyTorch wraps Python scalars automatically for ergonomics: writing r * 0.99 is clearer than r * tensor(0.99), and the wrapping is cheap because the scalar is a single element. The cost of the requirement would be that every scalar constant in a tensor expression — the learning rate, the discount, the temperature — would have to be constructed as a tensor on the correct device before use, and the learner would have to keep the scalars’ devices in sync with the tensors’. The automatic wrapping removes both burdens, at the cost that scalar promotion is another silent behavior the learner must know about.

WE-L.3.b — Completion problem

Input. Per-environment rewards r of shape (32, 1) (a column) and per-environment advantages adv of shape (32,) (a vector). The learner writes weighted = r * adv.

Trace (final 25% as prompts).

1. r shape (32, 1), adv shape (32,).
2. Align from the right: r is (32, 1), adv is treated as (1, 32).
3. For the trailing axis: 1 vs. 32 — compatible, expands to 32. ⟨your turn: what does the other axis contribute?⟩
4. For the leading axis: 32 vs. the implicit 1 — ⟨your turn: compatible or not, and what is the result?⟩

Answers.

1. The other axis contributes 32; it does not expand.
2. The leading axis is 32 vs. an implicit 1, which is compatible and expands to 32; the overall result shape is (32, 32).

The lesson. The learner intended a per-environment scalar weighting and produced a 32×32 outer-product-like tensor. The next operation — a loss, a sum, a backward — will consume the (32, 32) tensor and will not complain, because (32, 32) is a legal shape for a loss. The result is a policy trained against the wrong objective. The fix is r.squeeze(-1) * adv (or r * adv.unsqueeze(-1)), and the assertion that the result has shape (32,) is what would have caught it.

WE-L.3.c — Faded: the running case with one parameter changed

Setup. The 6-DOF arm has a per-environment reward of shape (8,) across 8 environments and a per-environment, per-joint action of shape (8, 6). The learner wants to weight each environment’s action by its reward, producing an (8, 6) tensor.

Question. Write the broadcast expression, state the intermediate shape after alignment, state the result shape, and state what shape assertion would catch an error.

You must produce the intermediate steps. The setup and the answer are given.

Answer. r.unsqueeze(-1) * a with r of shape (8,) and a of shape (8, 6). Intermediate: r.unsqueeze(-1) has shape (8, 1), aligned against (8, 6) — trailing axis 1 vs. 6 expands to 6, leading axis 8 vs. 8 matches, result (8, 6). The assertion is assert (r.unsqueeze(-1) * a).shape == (8, 6). The learner who wrote r * a directly: r is (8,), a is (8, 6), alignment gives (1, 8) against (8, 6), and the trailing 8 against 6 raises (verified: The size of tensor a (8) must match the size of tensor b (6)). No rank-1 operand can broadcast a rank-2 result up to rank 3, so this particular mistake is loud rather than silent.

The silent variant of the same mistake is a rank coincidence on the trailing axis. Suppose the learner holds a per-joint reward vector r of shape (6,) — one value per joint — and an action tensor a of shape (8, 6) — eight environments, six joints — and writes r * a intending a per-environment weighting. It runs, returns (8, 6), and multiplies every environment by the same per-joint vector (verified). The result is legal, the shape is exactly the one wanted, and the semantics are wrong: joint $j$ of environment $e$ was scaled by a quantity that has nothing to do with environment $e$. The general rule: a rank-1 operand always aligns against the last axis, so the silent case is the one where the rank-1 length equals the last axis’s extent — here 6 joints — and the loud case is the one where it equals the first axis’s extent, as in r(8,) * a(8,6), which raises on the trailing mismatch.

The lesson. Broadcasting compatibility is a property of the shapes, not of the meaning; the same expression is safe or unsafe depending on whether the numeric sizes happen to coincide. The assertion is the only thing that distinguishes the two cases.

WE-L.3.d — Contrastive near-miss: the intended broadcast against the silent one

Correct solution. The learner has logits of shape (32, 10) for a 10-way action discretization and a per-action bias of shape (10,). They write logits + bias. Result shape (32, 10), with the bias added to each row. Correct.

Near-miss. The learner has per-environment rewards of shape (32,) and per-environment losses of shape (32,), and writes rewards * losses.unsqueeze(-1) — intending (32,) * (32, 1) to produce (32, 1), which they then sum over the leading axis. The broadcast produces (32, 32); .sum() with no argument reduces to a scalar, which is the sum over all 1,024 (i, j) pairs rather than the intended sum over i. (Had they written .sum(0) or .sum(1) they would have got a (32,), and the bug would have been half-hidden: a legal shape, wrong contents.) The loss is larger or smaller depending on the data, the gradient is wrong, and training still decreases a loss — a different loss than intended.

Why the near-miss produces a plausible but incorrect number. Every step is a legal tensor operation with a legal shape: (32,) * (32, 1) → (32, 32), .sum() → scalar. The loss decreases over training. The policy learns something, and the something is the objective the learner accidentally wrote. The discrepancy only becomes visible if the learner compares against a reference implementation, or if the policy’s behavior is evaluated on the target task — at which point the wrong objective produces a policy that optimizes the wrong quantity.

The discrepancy, stated. The correct expression (rewards * losses).sum() has shape sequence (32,) * (32,) → (32,) → scalar; the near-miss (rewards * losses.unsqueeze(-1)).sum() has shape sequence (32,) * (32, 1) → (32, 32) → scalar. The two scalars are different sums, and the difference is exactly the sum of the off-diagonal terms of the outer product. The learner who printed only the final scalar saw no difference in shape; the learner who printed the shape at each step saw the (32, 32) immediately.

Self-explanation 7. Before reading on: in the near-miss, what single additional print statement would have caught the bug, and at which step?

Answer. Printing the shape of rewards * losses.unsqueeze(-1) before the sum. The step is the multiplication, and the printed (32, 32) is not the intended (32, 1) or (32,); the learner would have seen the inflation immediately. Printing only the post-sum scalar would not have caught it, because (32, 32).sum() and (32,).sum() are both scalars.

§L.3.3 — Comparative table: alignment strategies

Design principle (P1.3). Introduce rank, then align. When two tensors of different rank are combined, the learner should state the rank of each and the axis on which they are meant to align before writing the expression, and should use unsqueeze to make the alignment explicit rather than relying on the implicit (B,) ↔ (1, B) alignment. The explicit form costs one character and removes an entire bug class.

§L.3.4 — Concept check

Concept check (displaces M2). A learner says: “If the shapes are compatible, the operation is correct.” Give a pair of shapes that are compatible but for which the result is almost certainly not what the learner intended, and name the rule the learner has conflated.

Answer. (B, 1) and (B,) are compatible — they broadcast to (B, B) — and for the overwhelming majority of robot-learning operations, a (B, B) result from two B-sized inputs is not what the learner intended. The learner has conflated compatibility (the framework’s acceptance condition) with intention (the programmer’s meaning), and the two are different relations: compatibility is a property of the shape pair, intention is a property of the operation’s role in the program.

§L.3.5 — Consolidation checkpoint 3

(a) You should now be able to: (i) apply the right-aligned broadcasting rule to any pair of shapes and state the result shape; (ii) identify the intended-broadcast case (result shape equals one operand’s shape) and the unintended-broadcast case (result shape is larger than both operands); (iii) choose between implicit broadcast, explicit unsqueeze, expand, and assertion, and state the cost and signal of each.

(b) Transfer probe. A vision policy produces a per-image feature of shape (B, D) and a per-image, per-patch feature of shape (B, N, D). The learner wants to compute a cosine similarity between the image feature and each patch feature, producing (B, N). Which of the following expressions is correct? (a) (feat.unsqueeze(1) * patches).sum(-1), (b) (feat * patches).sum(-1), (c) (feat.unsqueeze(2) * patches).sum(-1). Answer sketch: (a). feat.unsqueeze(1) has shape (B, 1, D), which broadcasts against (B, N, D) to (B, N, D), and the sum over the last axis gives (B, N). (b) aligns (B, D) against (B, N, D) as (1, B, D) vs. (B, N, D), which is incompatible (B vs. N) unless B = N, in which case it silently computes the wrong product. (c) inserts a size-1 axis at the end, giving (B, D, 1), which is incompatible with (B, N, D).

(c) Remediation rule. If you cannot do (b), return to §L.3.2 and redo WE-L.3.b, then re-attempt (b) by writing down the shape of each expression at each step.

Part D — Device Management

§L.4 — Objects: device, transfer, retention

Device is the memory space in which a tensor’s data buffer resides. The two common devices are cpu and cuda (with an optional index, cuda:0, cuda:1, for multi-GPU systems). A tensor’s device is set at creation and changed only by an explicit move (.to(device), .cuda(), .cpu()).

Device mismatch is the condition in which two tensors on different devices are combined in an operation. It raises a RuntimeError with a message of the form Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!. Mismatch is the loud device failure.

Device retention is the condition in which a live tensor on a GPU is stored in a Python object — a list, a dict, a logging buffer — that persists across training iterations. The tensor itself is small; what it can keep alive is not. Whether anything large is kept alive depends on when the tensor was appended relative to .backward(), and this is the part that is usually got wrong. Appending before .backward() keeps the whole ancestry alive: every intermediate tensor saved for the backward pass is still referenced, and the memory grows by the graph’s size each iteration. Appending after .backward() keeps only the empty grad_fn shells — the saved tensors have already been freed, and touching one raises RuntimeError: … or directly access saved tensors after they have already been freed. That costs a few kilobytes per iteration, not megabytes. Retention is the silent device failure, and the canonical training loop appends after .backward(), so the version that OOMs is the one that accumulates a list of losses and does its backward later, or calls .backward() on only the last element of a batch of accumulated losses.

The formal distinction is between the value of a tensor (a handful of floats), its graph ancestry (every intermediate tensor saved for the backward pass), and the shell that survives after the ancestry is freed. .item() extracts the value and drops both; .detach() drops the ancestry and the requirement to track it; .to('cpu') moves the value while the source’s ancestry lives until the source is garbage-collected. In a logging context, .item() is the correct operation, because the log only needs the value — and it is correct for a second reason, that it is the only one of the three that guarantees the log cannot hold the graph.

§L.4.1 — Result: device mismatch raises; device retention does not

Claim. Of the two classes of device bug, mismatch is caught by the framework and retention is not; consequently, a training loop that runs without error is not evidence that its device management is correct.

Proof sketch. Mismatch is detected at the operation’s argument-checking stage, before any computation is dispatched, and the framework raises. Retention is not detected at all, because retaining a tensor in a Python list is a legal Python operation, and the graph it belongs to is kept alive by Python’s reference counting rather than by any framework-managed resource. The OOM that eventually results is attributed to the loop’s iteration count, not to the specific list that is retaining the tensors, and the learner who debugs it sees only that the process dies after N iterations — not that a losses.append(loss) line is the cause.

Practical corollary. The learner should adopt a rule rather than an intuition: every tensor that is stored for later — in a log, a buffer, a metric accumulator — is stored as a Python scalar via .item(), not as a tensor. The rule is mechanical and easy to enforce in code review; the intuition that “small tensors are fine to store” is wrong, because the tensor’s graph is not small.

§L.4.2 — Worked-example ladder

WE-L.4.a — Fully worked: determining device at each step

Input. A training loop on a machine with one GPU. The model is constructed on cpu by default, the data is loaded on cpu by the data loader, and the learner moves both to GPU before the forward pass.

Trace.

1. device = cuda:0 — a torch.device object, used as the target for all subsequent moves.
2. model = PolicyNet() — parameters created on cpu, because nn.Module constructs its parameters on the default device unless told otherwise.
3. model = model.to(device) — every registered parameter and buffer is moved to cuda:0 in place; the module’s parameters() iterator now yields cuda:0 tensors.
4. for x, y in loader: — x and y are created by the data loader on cpu, because the data loader does not know about device and cannot, without the pin_memory/non_blocking machinery that we will discuss in step 5.
5. x = x.to(device, non_blocking=True) — moves to GPU; with pinned host memory, the transfer is asynchronous and can overlap with the previous iteration’s compute.
6. y = y.to(device, non_blocking=True) — same.
7. pred = model(x) — the forward pass executes on GPU, because both the parameters and the input are on GPU; all intermediate tensors are on GPU.
8. loss = loss_fn(pred, y) — the loss is on GPU.
9. optimizer.zero_grad(); loss.backward(); optimizer.step() — the backward pass and the update execute on GPU; the parameter updates are applied to the cuda:0 parameter tensors.
10. losses.append(loss.item()) — .item() extracts the Python float and drops the graph; the list retains floats, not tensors.

Output and interpretation. Every step in the trace has a defined device, and the device is the one the operation requires. The learner who skipped step 3 and left the model on cpu would raise at step 7, because x is on GPU and the parameter is on CPU — the good case. The learner who skipped .item() at step 10 would append a tensor after .backward() has already run, so what accumulates is the grad_fn shell rather than the saved tensors: measured at about 10 KB per iteration at a width of 128, and flat in the model’s width, against 71 KB per iteration when the tensors are appended before the backward pass. Neither reaches an OOM within thousands of iterations for a small model, and that is the point — the leak is real, silent, and slow, which is why it is diagnosed by watching RSS climb rather than by a crash. The version that OOMs quickly is the one that keeps the losses and defers the backward pass, or calls .backward() only on the last of several accumulated losses; then the whole ancestry of each earlier loss is still referenced.

Self-explanation 8. Before reading on: in step 5, why is non_blocking=True only beneficial when the source data is in pinned memory, and what does “pinned” mean?

Answer. “Pinned” (page-locked) memory is host memory that the operating system has promised not to page out, so a GPU can read it via direct memory access without a driver-managed staging buffer. The transfer from pinned memory can therefore be initiated asynchronously and overlap with compute, which is what non_blocking=True requests. A transfer from pageable memory must first be copied into a driver-managed pinned buffer, and that copy is synchronous, so non_blocking=True on a pageable source has no effect and can silently produce a race condition if the source is later modified.

WE-L.4.b — Completion problem

Input. A validation loop inside a training loop, with the model on GPU and the validation loader on CPU.

Trace (final 25% as prompts).

1. model.eval() — mode change only; device unchanged.
2. with torch.no_grad(): — graph construction disabled; device unchanged.
3. for x, y in val_loader: — x, y on cpu.
4. x = x.to(device); y = y.to(device) — moves to GPU.
5. pred = model(x); val_loss += loss_fn(pred, y).item() — ⟨your turn: what is the device of val_loss, and why?⟩
6. model.train() — ⟨your turn: which device is the model on, and has the device changed during validation?⟩

Answers.

1. val_loss is a Python float, because .item() extracted the scalar from the GPU tensor and dropped the graph. The device of val_loss is not “the device of the tensor” — the tensor’s device is cuda:0, but val_loss is a Python object with no device.
2. The model remains on cuda:0; model.eval() and model.train() do not change the device, only the training flag. The device is preserved across mode changes and across validation loops, and the only operation that changes it is .to(device).

The lesson. eval() and train() are mode operations; .to(device) is a device operation; .item() is a value operation; the four are orthogonal, and a learner who conflates them will introduce bugs in all four categories. The most common conflation is between eval() and no_grad() — they are both “for evaluation” in intent and both must be called, but for different reasons, and the learner who calls only one has fixed half the problem.

WE-L.4.c — Faded: the running case with one parameter changed

Setup. The 6-DOF arm is simulated across 16 environments on a workstation with two GPUs, and the learner wants to use both — the model on cuda:0, the simulation on cuda:1, with observations transferred from cuda:1 to cuda:0 before the policy.

Question. The learner writes obs.to('cuda:0') before the forward pass. What is the device of obs after the .to call, and what is the device of the observation buffer held by the simulator after the call?

You must produce the intermediate steps. The setup and the answer are given.

Answer. After .to('cuda:0'), obs is a tensor on cuda:0. The simulator’s original buffer is unchanged and remains on cuda:1 — .to(device) returns a new tensor on the target device and does not mutate the source, which is one of the few PyTorch operations that is not in-place. The learner who expected the simulator to see the moved tensor has conflated .to(device) with an in-place move; the correct in-place form is obs = obs.to('cuda:0') (rebinding the variable) or obs.data = obs.to('cuda:0').data (mutating the buffer, which is rarely what you want and is easy to get wrong).

The lesson. .to(device) is functional, not in-place; it returns a new tensor and leaves the source untouched. The in-place variants are .cuda(), .cpu(), and .to(device, inplace=True) on some operations, and they are a different class of operation with a different failure mode (mutating a source that other code holds a reference to).

WE-L.4.d — Contrastive near-miss: loss.item() vs. loss

Correct solution. The training loop appends loss.item() to the losses list, and the list contains Python floats. The graph for each loss is released after the backward pass, and the GPU memory profile is flat across iterations.

Near-miss. The training loop appends loss — the GPU tensor — to the losses list. The list contains live GPU tensors, each holding a device-side scalar and a grad_fn shell. If the append happens before the backward pass, the shell still references the saved tensors and the list retains the whole ancestry; if it happens after, only the shell survives and the growth is a few kilobytes an iteration. Either way the tensors are on the GPU and stay there, and the memory climbs monotonically until the process is killed with an out-of-memory error that points at a torch.empty or torch.zeros call deep inside the model’s forward pass, not at the losses.append line.

Why the near-miss produces a plausible but incorrect diagnosis. The OOM traceback names the allocation that failed — usually a batch’s activation — rather than the retention that caused the failure. The learner sees “the forward pass allocated too much” and reduces the batch size, which delays the OOM but does not fix it. The actual fix is to append loss.item() rather than loss, and the actual cause is the retention of the graph, not the size of the batch. The near-miss’s symptom (OOM) and the correct solution’s symptom (no OOM) are both consistent with “the batch is too large” if the retention is not considered.

The discrepancy, stated. loss.item() has type float and no device; loss has type Tensor and device cuda:0 and, more importantly, holds a grad_fn. The two differ in exactly one property that matters here — whether anything large is reachable from the object through that grad_fn — and that property is not visible in print(loss) or loss.shape or loss.dtype. The learner who inspects the object by its printed repr sees no difference; the learner who knows that .item() is a graph-dropping operation as well as a scalar-extracting one knows to call it.

Self-explanation 9. Before reading on: in the near-miss, what is the minimal change to the logging line that fixes the OOM while preserving the ability to average the losses later?

Answer. losses.append(loss.detach().item()) — the .detach() is redundant if .item() is called (the item extraction drops the graph whether or not the tensor was detached), but it is not harmful; the minimal fix is losses.append(loss.item()), and the averaging is sum(losses) / len(losses) on Python floats. The learner who wanted the running mean without storing every loss can instead accumulate running += loss.item().

§L.4.3 — Comparative table: device-related operations

Design principle (P1.4). Log scalars, not tensors. Every quantity that is stored for later inspection — a loss, a reward, a success rate — is stored via .item() (or via a NumPy conversion after .detach().cpu(), which also drops the graph), never as a live tensor. The rule is mechanical, reviewable, and eliminates the device-retention class of bug in its entirety.

§L.4.4 — Concept check

Concept check (displaces M4). A learner says: “I have never had a device bug, because every device mismatch raises and I fix it immediately.” Which class of device bug has the learner not encountered, and what distinguishes it from the one they have?

Answer. The learner has encountered device mismatch, which raises at the operation and is fixed by moving a tensor. The learner has not encountered device retention, which does not raise: it accumulates graph memory in a persistent container and manifests as an out-of-memory error attributed to an unrelated allocation. The two are distinguished by the fact that mismatch is detected by the framework at the operation, and retention is not detected at all — it is detected only by the operating system, after the fact, with a traceback that points at the wrong line.

§L.4.5 — Consolidation checkpoint 4

(a) You should now be able to: (i) determine the device of every tensor in a training loop from its creation and move operations; (ii) distinguish .to(device), .detach(), and .item() by their effect on the device, the graph, and the source; (iii) diagnose an OOM that is caused by retention rather than by batch size.

(b) Transfer probe. A learner’s training loop runs for 800 iterations on a 24 GB GPU and then OOMs, every time, regardless of the batch size. They have tried batch sizes from 8 to 256 and the OOM iteration count scales roughly inversely with the batch size. What is the most likely cause, and what is the minimal diagnostic? Answer sketch: device retention of a live tensor in a list that grows by one entry per iteration; the graph size per iteration scales with batch size, so the number of iterations to OOM scales inversely with batch size — which is the signature. The diagnostic is to print(len(losses), losses[0].dtype) and to check whether losses[0] is a Tensor rather than a float; if it is a tensor, the fix is .item().

(c) Remediation rule. If you cannot do (b), return to §L.4.2 and redo WE-L.4.d, then re-attempt (b) by considering which quantity in the loop grows with the iteration count.

Part E — Failure Taxonomy, Lab, Cheat Sheet, Exercises, Exit Test

§L.5 — Failure taxonomy

Before the lab, a table of the symptoms the learner will observe, the three most likely causes for each, the discriminating test that identifies the cause, and the fix. This is the lesson’s debugging map, and it covers every break-it step in the lab plus two failures not exercised there.

The two failures not exercised in the lab are the “CPU tensor in a GPU loop” and the “non-contiguous view” cases; both appear in the exercises.

§L.6 — The lab

§L.6.1 — Task

Build a tensor-sanity harness for the running case — a 6-DOF arm with 8 parallel environments and a 50-step rollout — and use it to reproduce, predict, and fix three silent tensor bugs. The harness is a small set of functions that the learner will reuse in Lessons 0.1.2–0.1.4 and in Phase 1.

§L.6.2 — Conditions

A machine with Python 3.10+ and PyTorch 2.2+. No GPU required; the device experiments run on CPU with device = torch.device('cpu') if no GPU is available, and the retention experiment uses a small enough tensor that CPU memory suffices. All runs use torch.manual_seed(42). Runtime budget: 15 minutes total for all twelve steps.

§L.6.3 — Controls held fixed

The seed is 42. The shapes of the running-case tensors are fixed at (8, 50, 6) for observations and (8, 50, 6) for actions. The batch size in the broadcast experiment is fixed at 32. The retention experiment uses a graph whose depth is fixed by a two-layer MLP of width 64.

§L.6.4 — Metrics table

§L.6.5 — Pre-registered hypotheses

State each of the following, and predict the number, before running any code.

• H1. On the correct pipeline, the shape-correctness count will be 12 of 12 (twelve assertions in the harness, each corresponding to a tensor whose shape is stated in the trace).
• H2. Appending live tensors instead of scalars raises no exception at all over 2,000 iterations, and the retained graph is visible only as a growing len(logs) and a non-float type(logs[0]). Measured here: 2,000 iterations of a two-layer width-64 MLP at batch 32 on CPU reached 255 MB peak RSS and did not approach OOM. The prediction is therefore not an iteration count; it is that no OOM occurs within 2,000 iterations on CPU, and the falsifier is an OOM before that count. On a GPU with a large graph the count would be smaller, which is why the metric is the retained-graph type, not the iteration count.
• H3. Of the three break-it steps, all three will raise no exception at the site of the bug; the prediction is the set of steps that are silent, namely {step 7, step 8, step 9}. Step 7 is the one that surprises: the missing batch axis is a rank bug, not a width bug, and nn.Linear checks only the width.
• H4. The device-mismatch raise rate is 3 of 3 on a machine with a GPU, and 0 of 3 on a CPU-only machine, because the harness builds device from torch.cuda.is_available() and on a CPU-only host device is cpu, so no mismatch is ever constructed. The prediction must be stated conditionally, and the CPU-only run must report 0/3 with the reason, not treat it as a failure of the criterion.

§L.6.6 — The break-it sequence

Three deliberate corruptions, ordered by subtlety. The learner writes down the predicted symptom before running the corruption, then runs it and records the actual symptom.

1. The missing batch axis. Remove the unsqueeze(0) from a single-observation inference call. Predicted symptom: no exception at the linear layer — nn.Linear(6, 6) accepts a (6,) input and returns (6,), so the bug survives the forward pass. The divergence is in the rank of the result, and it becomes visible one of two ways: a ValueError: expected 2D or 3D input (got 1D input) if a BatchNorm1d sits downstream, or a silently wrong shape if the caller does torch.cat(..., dim=0) or a dim=0 reduction. Actual symptom: no exception at the linear layer; pred.mean(0) returns a scalar rather than a (6,) vector. This is a silent case at the point of the bug, which is the opposite of what the intuition “wrong shape means a crash” predicts.
2. The unintended broadcast. Change (rewards.unsqueeze(-1) * advs).sum() to (rewards * advs.unsqueeze(-1)).sum() in the loss computation, with rewards and advs both of shape (32,). Predicted symptom: no exception; the loss value is different from the correct one by the sum of the off-diagonal terms; the shape at the multiply step is (32, 32) rather than (32,). Actual symptom: as predicted, silent — no exception.
3. The retention leak. Change losses.append(loss.item()) to losses.append(loss) and run for 2,000 iterations or until OOM, printing torch.cuda.memory_allocated() (or tracemalloc on CPU) every 100 iterations. Predicted symptom: no exception until the OOM; the memory grows linearly with the iteration count; the OOM traceback points at an allocation in the forward pass, not at the append. Actual symptom: as predicted, silent until the OOM.

All three are silent at the site of the bug. Step 1 fails loudly only if a BatchNorm1d is present downstream; without one, the rank error propagates as a wrong shape. This is the lesson’s central claim in miniature: the bugs that matter are the ones the framework does not catch.

§L.6.7 — Acceptance criteria

§L.6.8 — What good looks like

A strong submission has five properties:

1. The predictions are written before the runs. The notebook has a timestamp or a separate file mtime for the predictions and the actuals; the predictions are specific numbers or sets, not “it will probably fail.”
2. The silent bugs are identified as silent. The learner states, for each break-it step, whether the framework raised or not, and why — not just that the bug was present.
3. The retention diagnosis distinguishes cause from symptom. The learner writes that the OOM’s traceback points at the forward pass but the cause is the retention in the list, and the fix is .item(), not a batch-size reduction.
4. The fixes are minimal. Each fix is one line (or one function call), and the learner states why the minimal fix is sufficient rather than adding defensive code around it.
5. The harness is reusable. The shape-assertion functions are written as reusable functions with a single entry point, not as inline assert statements scattered through the notebook, because the learner will use them in the next three lessons.

§L.6.9 — Runnable lab code

The following is complete, seeded, and annotated. It is the only runnable code in the lesson.

# lab_0_1_1.py — Tensor sanity harness for the 6-DOF, 8-env, 50-step running case
# Requires: Python 3.10+, PyTorch 2.2+

import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(42)                                    # fix the seed for reproducibility

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"device = {device}")

# --- Step 1: the running-case observation tensor -----------------------------

num_envs, rollout_len, obs_dim = 8, 50, 6
obs = torch.randn(num_envs, rollout_len, obs_dim, device=device)
print(f"obs.shape = {obs.shape}, strides = {obs.stride()}, device = {obs.device}")
# Expected: shape (8, 50, 6), strides (300, 6, 1), device as above

# --- Step 2: flat offset of (3, 17, 4) --------------------------------------
offset = 3 * obs.stride(0) + 17 * obs.stride(1) + 4 * obs.stride(2)
print(f"flat offset of (3, 17, 4) = {offset}")           # Expected: 1006

# --- Step 3: view vs copy ---------------------------------------------------
reshaped = obs.reshape(num_envs, rollout_len * obs_dim)
print(f"reshape shares buffer? {reshaped.data_ptr() == obs.data_ptr()}")  # Expected: True
reshaped[0, 0] = 99.0
print(f"obs[0, 0, 0] after writing to view = {obs[0, 0, 0].item()}")       # Expected: 99.0

cloned = obs.clone()
cloned[0, 0, 0] = -1.0
print(f"obs[0, 0, 0] after writing to clone = {obs[0, 0, 0].item()}")      # Expected: 99.0 (unchanged)

# --- Step 4: permute and view ----------------------------------------------
permuted = obs.permute(2, 0, 1)                          # shape (6, 8, 50), non-contiguous
print(f"permuted.shape = {permuted.shape}, strides = {permuted.stride()}")
print(f"permuted.is_contiguous() = {permuted.is_contiguous()}")  # Expected: False
try:
    permuted.view(-1)
except RuntimeError as e:
    print(f"view raised as expected: {e}")               # Expected: raised
flat_copy = permuted.reshape(-1)                        # falls back to copy; silent
print(f"reshape succeeded, shape = {flat_copy.shape}")   # Expected: (2400,)

# --- Step 5: intended broadcast (bias) -------------------------------------
batch, num_actions = 32, 10
logits = torch.randn(batch, num_actions, device=device)
bias = torch.randn(num_actions, device=device)
out = logits + bias
print(f"logits + bias -> {out.shape}")                   # Expected: (32, 10)
assert out.shape == (batch, num_actions)

# --- Step 6: unintended broadcast (the (B,1) x (B,) trap) -------------------
rewards = torch.randn(batch, 1, device=device)          # column
advs = torch.randn(batch, device=device)                # vector
bad = rewards * advs
print(f"rewards * advs -> {bad.shape}")                 # Expected: (32, 32) — the silent bug
assert bad.shape == (batch, batch)                      # this assertion passes — and reveals the bug

good = rewards.squeeze(-1) * advs                       # the intended (32,) result
print(f"rewards.squeeze(-1) * advs -> {good.shape}")    # Expected: (32,)
assert good.shape == (batch,)

# --- Step 7: break-it #1 — the missing batch axis (silent) -----------------
policy = nn.Linear(obs_dim, obs_dim).to(device)
single_obs = obs[0, 0]                                  # shape (6,) — a single observation
pred_unbatched = policy(single_obs)                     # NO raise: (6,) -> (6,)
print(f"policy((6,)) -> {pred_unbatched.shape}")        # Expected: torch.Size([6]) — silent
pred_batched = policy(single_obs.unsqueeze(0))          # (1, 6) -> (1, 6)
print(f"policy((1,6)) -> {pred_batched.shape}")         # Expected: torch.Size([1, 6])
print(f"values identical: "
      f"{torch.allclose(pred_unbatched, pred_batched.squeeze(0))}")   # True — no signal at all

# The rank difference is what breaks downstream. A dim=0 reduction silently
# returns a scalar instead of a per-feature vector:
print(f"pred_unbatched.mean(0).shape = {pred_unbatched.mean(0).shape}")  # torch.Size([])
print(f"pred_batched.mean(0).shape   = {pred_batched.mean(0).shape}")    # torch.Size([6])

# A BatchNorm1d downstream is the loud case, and it names the wrong layer:
try:
    nn.BatchNorm1d(obs_dim)(pred_unbatched)
except ValueError as e:
    print(f"BatchNorm1d on (6,) raised: {e}")            # Expected: expected 2D or 3D input

action = policy(single_obs.unsqueeze(0)).squeeze(0)      # the fix
print(f"action.shape = {action.shape}")                  # Expected: (6,)

# --- Step 8: break-it #2 — the unintended broadcast (silent) ---------------
rewards = torch.randn(batch, device=device)
advs = torch.randn(batch, device=device)
loss_wrong = (rewards * advs.unsqueeze(-1)).sum()       # silent (32, 32) -> scalar
loss_right = (rewards * advs).sum()                     # correct (32,) -> scalar
print(f"loss_wrong = {loss_wrong.item():.4f}, loss_right = {loss_right.item():.4f}")
# The two are different; neither raised; the shapes at the multiply step were (32,32) vs (32,)
print(f"loss_wrong.shape after sum = {loss_wrong.shape}")  # Expected: torch.Size([]) — a scalar, no signal

# --- Step 9: break-it #3 — the retention leak (silent until OOM) -----------
def run_retention_experiment(append_tensors: bool, max_iters: int = 2000):
    """Run a small training loop, logging losses either as tensors or as floats."""
    model = nn.Sequential(nn.Linear(64, 64), nn.ReLU(), nn.Linear(64, 1)).to(device)
    opt = torch.optim.SGD(model.parameters(), lr=0.01)
    log = []
    for i in range(max_iters):
        x = torch.randn(32, 64, device=device)
        y = torch.randn(32, 1, device=device)
        opt.zero_grad()
        loss = F.mse_loss(model(x), y)
        loss.backward()
        opt.step()
        if append_tensors:
            log.append(loss)                            # retention: the graph is kept alive
        else:
            log.append(loss.item())                     # correct: a float is stored
    return log

# Run the correct version; it completes.
log_correct = run_retention_experiment(append_tensors=False)
print(f"correct log type = {type(log_correct[0])}, len = {len(log_correct)}")

# Run the retention version; on a small machine, cap the iterations to avoid a real OOM.
# If you have memory headroom, set max_iters=2000 and observe the OOM and its traceback.
log_buggy = run_retention_experiment(append_tensors=True, max_iters=500)
print(f"buggy log type = {type(log_buggy[0])}, len = {len(log_buggy)}")
# On a machine with enough headroom, increase max_iters until the process is killed,
# and observe that the traceback points at an allocation in the forward pass, not at append.

# --- Step 10: shape assertions as a reusable harness -----------------------
def assert_shape(t: torch.Tensor, expected: tuple, name: str = "tensor"):
    """Fail loudly with a useful message if t.shape != expected."""
    if t.shape != expected:
        raise AssertionError(f"{name}: expected shape {expected}, got {t.shape}")
    return t

# Demonstrate on the correct pipeline
assert_shape(obs, (8, 50, 6), "obs")
assert_shape(logits, (32, 10), "logits")
assert_shape(out, (32, 10), "logits+bias")
assert_shape(good, (32,), "rewards*advs")
print("all assertions passed on the correct pipeline")

# --- Step 11: device-mismatch raise rate -----------------------------------
# A mismatch can only be constructed if a second device exists. On a CPU-only
# host, `device` is cpu and every operand is already there, so the honest report
# is 0/3 with a reason -- not a failed criterion.
other = torch.device("cuda:0") if torch.cuda.is_available() else None
cpu_t = torch.randn(4, 4)                               # created on cpu
gpu_t = torch.randn(4, 4, device=device)                # created on the target device
if other is None:
    print("device-mismatch raise rate: n/a -- only one device present (cpu)")
    raises = 0
else:
    other_t = torch.randn(4, 4, device=other)           # genuinely on a second device
    mismatches = [("cpu + other", cpu_t, other_t),
                  ("cpu @ other", cpu_t, other_t),
                  ("cpu * other", cpu_t, other_t)]
    raises = 0
    for name, a, b in mismatches:
        try:
            _ = a + b if "+" in name else (a @ b if "@" in name else a * b)
            print(f"{name}: did not raise")              # should not happen
        except RuntimeError as e:
            raises += 1
            print(f"{name}: raised — {e}")
    print(f"device-mismatch raise rate: {raises}/{len(mismatches)}")

# --- Step 12: the summary --------------------------------------------------
print("\n--- lab 0.1.1 summary ---")
print(f"device: {device}")
print(f"obs shape/strides: {obs.shape} / {obs.stride()}")
print(f"view shared buffer: {reshaped.data_ptr() == obs.data_ptr()}")
print(f"permuted contiguous: {permuted.is_contiguous()}")
print(f"break-it silent steps: #1 (batch axis), #2 (broadcast), #3 (retention)")
if other is None:
    print("device-mismatch raise rate: 0/3 (CPU-only, no mismatch constructible)")
else:
    print(f"device-mismatch raise rate: {raises}/3")

Run the file end to end. The output should match the expected comments at each step. If any step does not match, the failure taxonomy in §L.5 names the likely causes.

§L.7 — The cheat sheet

§L.8 — Exercises with solutions

E 0.1.1.1 — The flat offset. A tensor has shape (4, 3, 2) and strides (6, 2, 1). Compute the flat offset of index (2, 1, 1), and state which of the tensor’s elements is at that offset if the tensor was created contiguously.

S 0.1.1.1. Offset = 2·6 + 1·2 + 1·1 = 12 + 2 + 1 = 15. For a contiguous (4, 3, 2) tensor, the elements are in row-major order, so flat index 15 is the element at coordinates (15 // 6, (15 % 6) // 2, (15 % 6) % 2) = (2, 1, 1) — the same index, as it must be for a contiguous tensor. The lesson: the stride equation is the definition of the index-to-memory mapping; for a contiguous tensor, it is the identity in row-major order, and for a non-contiguous view it is a permutation.

E 0.1.1.2 — The non-contiguous flatten (transfer probe). A tensor x has shape (4, 6). A learner computes y = x.t() (shape (6, 4)) and then z = y.reshape(24). Is z a view or a copy? What is the relationship between z’s element at index k and x’s element at index (i, j)? Hint: think in terms of the permuted row-major order.

S 0.1.1.2. y = x.t() is a view with shape (6, 4) and strides (1, 6) (the swap of x’s (6, 1)). It is not contiguous. z = y.reshape(24) therefore falls back to a copy, laid out in row-major order for y’s shape — i.e., z[k] = y[k // 4, k % 4] = x[k % 4, k // 4]. The learner who expected z[k] = x.flat[k] (the row-major flatten of x) has produced the transpose-flattened version instead. The lesson: reshape on a non-contiguous tensor preserves the values but reorders them to the target’s row-major layout; the silent risk is that the reordering is not the reordering the learner wanted.

E 0.1.1.3 — The diagnostic exercise. A learner’s training loop prints a loss that decreases, but the policy’s success rate on a held-out task is near chance. The loss expression is (advantages * log_probs).sum() where both are (64,) tensors. The learner’s code computes advantages = rewards - rewards.mean(), and rewards has shape (64, 1). Name the bug, state the shape of advantages, and state the shape of the product.

S 0.1.1.3. The bug is an unintended broadcast: rewards.mean() on a (64, 1) tensor reduces to a scalar (mean over all elements), and rewards - scalar has shape (64, 1). The advantages tensor is (64, 1), not (64,). The product advantages * log_probs aligns (64, 1) against (64,) as (64, 1) vs. (1, 64), broadcasting to (64, 64). The .sum() reduces the (64, 64) matrix to a scalar — the sum over all (i, j) pairs — and the gradient is wrong by the cross terms. The fix is rewards = rewards.squeeze(-1) before computing advantages, and an assertion that advantages.shape == (64,). The lesson: a broadcast bug can hide behind a loss that decreases, because the wrong objective is still an objective; the symptom is a policy that trains against the wrong target, not a loss that fails to decrease.

E 0.1.1.4 — The diagnostic exercise. A learner’s validation loop runs, but the validation loss is systematically 0.3 higher than the training loss on the same data. The model contains nn.BatchNorm1d. What is the most likely cause?

S 0.1.1.4. The validation loop is running in training mode — the learner forgot model.eval() before the validation loop. Batch norm in training mode uses batch statistics and updates running statistics; in eval mode it uses the frozen running statistics. The training-mode validation computes a different normalization than the deployment-mode inference, and the difference appears as a systematic loss gap. The fix is model.eval() at the top of the validation function, and a model.train() at the top of the training function (this is the subject of Lesson 0.1.4). The lesson: the gap between training and validation loss is not always overfitting; it can be a mode-management bug, and the discriminator is whether the gap disappears when the model is evaluated in eval mode.

E 0.1.1.5 — The constructive exercise (transfer probe). Write, in pseudocode, a function shape_check(expr, expected) that evaluates expr (a tensor expression) and returns the tensor if expr.shape == expected, raising an AssertionError with the actual and expected shapes otherwise. Then state two places in a robot-learning pipeline where you would call it and why.

S 0.1.1.5.

function shape_check(expr, expected, name="tensor"):
    t = evaluate(expr)
    if t.shape != expected:
        raise AssertionError(f"{name}: expected {expected}, got {t.shape}")
    return t

Two call sites: (1) the loss expression before backward(), because an unintended broadcast in the loss is the single most expensive silent bug — the model trains, and the wrong objective is never visible in the loss value; (2) the action tensor before it is sent to the simulator or the actuator, because a wrong-shape action is the place where a tensor bug becomes a physical event. The first call site catches the bug at the training step; the second catches it at the deployment step, and the two together cover the two moments where a wrong shape is most consequential. The lesson: the assertion is the place where intention is stated, and the two highest-value places to state intention are the loss and the actuator command.

E 0.1.1.6 — The constructive exercise. Write, in pseudocode, a SafeLogger class that accepts tensors and stores Python floats, asserting that the input is a scalar (shape ()), and raising if it is not. Then state what the logger prevents.

S 0.1.1.6.

class SafeLogger:
    def __init__(self):
        self.values = []
    def log(self, tensor, name="metric"):
        if tensor.shape != ():
            raise AssertionError(f"{name}: expected a scalar, got {tensor.shape}")
        self.values.append(tensor.item())
    def mean(self):
        return sum(self.values) / len(self.values)

The logger prevents the device-retention bug in its entirety: it refuses to store a non-scalar tensor, so a learner who tries to log a (32,) tensor — the shape that arises from a missing sum or a wrong broadcast — is told immediately that the metric is not a scalar. The .item() call is the only path to storage, and the assertion that the input is a scalar is the precondition that makes .item() safe. The lesson: the fix for a silent bug is not a comment telling the reader not to make the mistake; it is an interface that makes the mistake impossible.

E 0.1.1.7 — The constructive exercise (transfer probe). A learner wants to compute, for each of 8 environments, the mean of a 50-step, 6-joint trajectory. The tensor is (8, 50, 6). Write the expression, state the result shape, and state which axis the mean is over.

S 0.1.1.7. traj.mean(dim=1) has shape (8, 6) — the mean is over the time axis (dim=1), one mean per (environment, joint) pair. The learner who wrote traj.mean(dim=0) computed the mean over environments, not over time, and the result has shape (50, 6). Neither raises; both are legal; the difference is whether the mean is per-joint-over-time (the intended) or per-timestep-over-environments (the unintended). The lesson: reductions have an axis argument, and the axis is part of the intention; an assertion that traj.mean(dim=1).shape == (8, 6) states the intention and catches the confusion.

E 0.1.1.8 — The diagnostic exercise. A learner’s code raises RuntimeError: view size is not compatible with input tensor's size and stride at a .view(64) call. The input is the result of a .permute(1, 0) on a (4, 16) tensor. What is the input’s shape and strides, and what is the minimal fix?

S 0.1.1.8. The input is y with shape (16, 4) and strides (1, 16); it is not contiguous. .view(64) requires contiguity and raises. The minimal fix is y.contiguous().view(64) — this copies y into a contiguous buffer and reinterprets it. The alternative y.reshape(64) succeeds silently with the same copy, but it removes the signal; the explicit contiguous().view(64) is preferred because it states that a copy is being made. The lesson: view raises where reshape silently copies, and the raise is information — it tells the learner that the tensor’s layout is not what they may have assumed.

§L.9 — Exit test

You are ready for Lesson 0.1.2 when you can, without notes:

1. Name the five attributes of a tensor and state what each is determined by.
2. For the expression ((x @ W) + b).relu(), with x: (32, 6), W: (6, 64), b: (64,), state the shape at each intermediate step and identify which operation is a broadcast.
3. Given a tensor of shape (4, 6) and strides (6, 1), state the strides of its transpose and the flat offset of index (2, 3) in the transpose.
4. Decide, for each of reshape, view, permute, clone, whether it returns a view or a copy on a contiguous input and on a permuted input.
5. State the broadcasting rule, and give one shape pair that is compatible and almost certainly unintended.
6. Compute the result shape of (32, 1) * (32,) and of (32, 1) * (32, 1), and state which one is an outer product and which is element-wise.
7. State the difference between tensor(x) and as_tensor(x), and give one situation in which the difference produces a silent bug.
8. State the difference between device mismatch and device retention, and name the operation that fixes retention.
9. State why loss.item() is preferred to loss in a logging list, in terms of the graph and not only in terms of the device.
10. Write the pseudocode for a shape_check function and name the two places in a robot-learning pipeline where you would call it.
11. Explain why y.contiguous().view(64) is preferred to y.reshape(64) after a permute, in terms of the signal each operation gives.
12. Given a training loop that OOMs after 800 iterations regardless of batch size, name the most likely cause and the minimal diagnostic.

If any of items 1–12 is not answerable without notes, return to the section that introduced it: item 1 to §L.1, item 2 to §L.2, item 3 to §L.1.2, item 4 to §L.2.3, item 5 to §L.3, item 6 to §L.3.2, item 7 to §L.1.3, item 8 to §L.4, item 9 to §L.4.2, item 10 to §L.8, item 11 to §L.2.3, and item 12 to §L.4.2.

§L.10 — Provenance and evidence tiers

What this lesson reproduces. Every hypothesis in §L.6.5 is executed by the lab in §L.6.9 on the reader’s own machine, and the numbers quoted in the text are the ones the lab prints. The lab is the evidence. Where a claim’s evidence is “the code above printed this”, the tier is A and the reproduction status is R.

What this lesson does not reproduce. The claim that broadcasting is the leading cause of silent shape bugs in PyTorch practice is a community observation, not a measurement, and no experiment in this lesson tests its frequency. It is tier C, status P — a claim, unreproduced — and it is load-bearing for the lesson’s emphasis, not for any number. The lesson’s own arithmetic does not depend on it: the witness pair (B, 1) and (B,) is constructible and its output is measured, so the existence of the bug is tier A regardless of how common it is.

The 0/3 device result is a finding, not a gap. On a CPU-only machine the device-mismatch hypothesis cannot fail, because no mismatch is constructible. The lab prints the count rather than assuming it. A reader on a GPU machine should see 3/3; a reader who sees 0/3 on a GPU machine has found a bug in the harness, not evidence about PyTorch.

No external sources are cited in this lesson, and none are needed. Every result is either a definition, an identity the reader can check by hand, or a behavior the lab measures. Where the lesson names a convention (the (B, 1) vs (B,) distinction), the convention is PyTorch’s documented behavior and is checked by running it rather than by citing documentation.