Lesson 4.1.2 — Choosing the Embodiment and the Evaluation

Week 32

Prerequisite lessons: 4.1.1 (defining the capstone), 2.3.4 (in-hand and dexterous manipulation), 3.4.1 (the reality gap as a spectrum), 3.3.4 (safety and constraints on LLM outputs), 3.1.4 (VLA fine-tuning), 1.1.1 (forward kinematics).

Prior results used in this lesson. From 4.1.1: the capstone’s three properties, the seven subsystems, the six metrics, the four axes, the in and out lists, and the risk register. Each is restated at the point of use. From 2.3.4: the dexterity count $d = 6 + n_q - \operatorname{rank}(J)$ and the fact that a parallel-jaw gripper is exactly actuated at a two-contact grasp. From 3.4.1: the four divergence axes and the twin-track’s tail. From 3.3.4: the four-layer stack’s catch rates and the residual-risk register. From 3.1.4: the decision conjunction and the acceptance’s lower-bound. From 1.1.1: the running arm’s geometry, its alternating $z$/$y$ axes and its $a = 0.2$ m links.

What this lesson hands forward. The embodiment’s choice and the evaluation’s frozen design — the axes, the sample sizes, the baselines, and the decisiveness criterion. Lesson 4.1.3 designs the held-out suite’s stratification against this design. Module 4.2 builds the seven subsystems on the chosen embodiment. Module 4.3 iterates against the frozen metrics. Module 4.4 defends the choice against the decisiveness criterion.

The running case in one sentence. The running arm, gripper, eye-in-hand camera, table, cube, bin, and divider are unchanged, and the capstone adds three distractor objects; this lesson asks whether that embodiment is the right one to build the capstone on, and which evaluation makes the capstone’s result evidence.

§L.0 — Lesson map, objectives, and misconceptions

Observable outcomes. By the end of this lesson you can, without notes: (1) name the four embodiment axes and give the running case’s value on each; (2) compare the running case against a 7-DOF redundant arm, a dexterous hand, and a mobile arm on the axes; (3) state why the evaluation’s choice is coupled to the embodiment’s through the metrics, the baselines, and the distributions; (4) build the metrics-baseline matrix and state the baseline’s role in the claim; (5) compute the held-out set’s size from the acceptance’s resolution and state why the budget constrains the resolution, not the size; (6) design the four-plus-one axes and their stratification; (7) state the decisiveness criterion and apply it to a choice; (8) write the “what would change my mind” test and the register’s entries; (9) freeze the running case’s embodiment and evaluation, with their artefacts; (10) state the cost of revisiting the choice after the held-out suite is frozen.

Misconception inventory.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

Q1. In 4.1.1, the capstone’s seven subsystems were perception, scene understanding, grasp planning, motion planning, control, safety, and transfer. Which of them is the embodiment’s subsystem — the one whose structure changes most when the embodiment changes?

Answer. The grasp planning and the control are the embodiment’s most sensitive: the grasp’s geometry, the gripper’s opening and force limits, and the controller’s computed-torque law’s parameters are all functions of the embodiment, while the perception and the safety are largely embodiment-agnostic. Used in §L.B.1 to explain why the embodiment’s choice changes the metrics that measure the grasp.

Q2. In 2.3.4, a hand with $n_q = 6$ joints and three point contacts had $d = 6 + 6 - 6 = 6$, no internal motions. What does that imply for the capstone’s task if the dexterous hand were chosen?

Answer. It implies that the hand cannot reorient the cube in the hand without breaking and re-making contact; a capstone built on it would need finger gaiting and would add a subsystem the running case does not have. Used in §L.A.2 to argue that the dexterous hand multiplies the evaluation’s failure modes.

Q3. In 3.4.1, the four divergence axes were dynamics, perception, actuation, and timing. Which of them changes most between a 6-DOF arm and a 7-DOF redundant arm?

Answer. The dynamics — the additional link’s mass, inertia, and friction add a dimension to the identification — and the actuation, because the extra joint’s actuator adds a delay and a gain to the model. The perception’s camera and the timing’s period are largely unchanged. Used in §L.A.2 to quantify the alternatives’ costs.

Q4. In 3.1.4, the decision conjunction required all necessary conditions to hold. What does the conjunction imply for the capstone’s evaluation design if the metrics’ set is expanded by one metric?

Answer. It implies that the new metric’s condition is necessary, so the acceptance’s failure’s probability rises because a metric’s miss is sufficient to fail the conjunction; the additional metric’s test set’s size and the acceptance’s risk are both costs. Used in §L.C.2 to argue that a metric without a decision is a cost.

Part A — The embodiment’s four axes

§L.A.1 — The axes

The embodiment is the robot’s physical structure: the arm’s DOF and geometry, the gripper’s mechanism and capability, the sensors’ suite and mounting, and the actuators’ properties. The capstone’s choice of embodiment is a choice among candidate structures, and the candidates differ along four axes.

Degrees of freedom and redundancy. The arm’s DOF relative to the task’s six: a 6-DOF arm is exactly actuated for a 6-DOF task, and a 7-DOF arm has one degree of redundancy that can be used for obstacle avoidance or joint-limit avoidance. The redundancy’s cost is the inverse kinematics’ non-uniqueness, which the 1.1.2 solver must resolve, and the control’s null-space’s additional design.

Contact richness. The gripper’s contact model: a parallel-jaw gripper has a two-contact grasp, exactly actuated at the grasp (the 2.3.4 §L.A.1 count), and a dexterous hand has three or more contacts and the internal-motion’s richness. The contact’s richness is the richness of the manipulation tasks the capstone can demonstrate and the richness of the failure modes the evaluation must measure.

Sensor suite. The perception’s channels: the eye-in-hand RGB-D camera, a wrist force-torque sensor, tactile arrays, and proprioception. The suite’s choice determines the perception subsystem’s metrics and the safety stack’s inputs; a missing force sensor changes the 2.3.3 momentum observer’s role from a complement to a necessity.

Simulation fidelity. The simulator’s ability to model the embodiment’s dynamics and contacts. A parallel-jaw gripper’s contact is the simplest; a dexterous hand’s multi-contact is the hardest, and the 3.4.1 §L.B.2’s perception axis’s uncertainty grows with the hand’s geometry’s complexity.

The four axes are not independent. The DOF’s redundancy adds the dynamics’ dimension; the contact’s richness adds the failure modes; the sensor’s suite adds the perception’s channels; the fidelity’s cost adds the transfer’s data. The axes are a decomposition for comparison, and the choice’s arithmetic is the sum along them.

§L.A.2 — The running case and the alternatives

The running case’s values.

Degrees of freedom. 6 DOF, exactly actuated for a 6-DOF task, with the alternating $z$/$y$ axes and equal links $a = 0.2$ m. The home configuration’s FK is $\text{Trans}_x(1.2)$.

Contact richness. A parallel-jaw gripper with opening 0–0.08 m, closing velocity 0.05 m/s, force up to 40 N, standard grip 5 N. The grasp is a two-contact point-with-friction contact, and the 2.3.4 count gives $d = 6 + n_q - \operatorname{rank}(J) = 6 + 1 - 3 = 4$ for the object’s controllable DOF at a two-contact grasp — a number that the exactly-actuated arm’s single gripper’s DOF does not change, because the gripper’s DOF is the opening, not a manipulator’s joint.

Sensor suite. The eye-in-hand RGB-D camera with $K = [[500,0,320],[0,500,240],[0,0,1]]$, 640×480, distortion $k_1 \approx -0.3, k_2 \approx 0.1$, plus the joint encoders and the momentum observer’s derived wrench. No dedicated force sensor.

Simulation fidelity. The MuJoCo model with the contact law $F_n = \max(0, k_c\phi + b_c\dot\phi)$ at $k_c = 10^5$ N/m and the identification’s interval. The perception axis is the largest uncertainty, per 3.4.1 §L.B.2’s 45%.

The alternatives.

A 7-DOF redundant arm. The extra joint gives one degree of redundancy for obstacle avoidance and joint-limit avoidance. The 2.3.2 planner’s clearance is easier to satisfy in the clutter, and the 1.1.2 IK’s non-uniqueness is resolved by a null-space’s cost. The cost is the additional link’s dynamics dimension, the extra actuator’s delay, and the identification’s data’s size.

A dexterous hand. The three-finger hand of 2.3.4 with 6 joints and three contacts. The in-hand reorientation is the new capability, and the 2.3.4 §L.A.1’s exact actuation is the constraint: the reorientation requires finger gaiting, which is a new subsystem the running case does not have. The cost is the multi-contact’s simulation’s fidelity — the hardest on the 3.4.1 §L.B.2’s axes — and the safety stack’s force limits’ new dimension.

A mobile arm. The arm on a base with the camera’s SLAM’s pose. The workspace’s expansion is the new capability, and the 2.1.5 SLAM’s drift is the new failure mode. The cost is the base’s localization’s data and the 3.3.4 stack’s new layer for the base’s collision.

The comparison’s table.

The table’s role is not a ranking; it is the cost’s decomposition. The running case is the cheapest on every axis, the dexterous hand is the most expensive on the contact and the fidelity, and the mobile arm is the most expensive on the perception and the safety.

The running case’s choice is justified by the cost’s decomposition and the evaluation’s conclusiveness, not by the hardware’s ceiling. The dexterous hand’s in-hand reorientation is a genuine capability, and a capstone built on it would demonstrate the 2.3.4 module’s result; the cost is that the capstone’s evaluation would need the multi-contact’s metrics, the gait’s sub-tasks, and the sim-to-real’s perception’s data, and the evaluation’s resolution — the number of episodes per axis that the acceptance needs — would be the budget’s constraint. The running case’s cheaper axes buy the evaluation’s resolution, and the mastery is in the evidence’s precision.

Self-explanation 1. Why is the cheaper embodiment’s evaluation more precise for the same budget?

Answer. Because the budget is finite, and the cheaper embodiment’s test episodes cost less per episode and support fewer axes’ metrics; the same budget’s episodes give a tighter confidence interval on the metrics that matter, and the tighter interval is the evidence’s precision.

§L.A.3 — The choice’s criterion

The embodiment’s choice is made by the decisiveness criterion: the embodiment that makes the capstone’s result evidence is the one whose evaluation’s acceptance is decidable within the budget. The criterion is not “which embodiment is most capable” and not “which is easiest”; it is “which embodiment’s evaluation’s acceptance is conclusive”.

The criterion’s three components are the 4.1.1 §L.A.2’s three properties applied to the embodiment’s choice.

The integration’s coverage. The embodiment must exercise the modules the capstone’s capability matrix names as in-scope. The running case exercises the perception, the reasoning, the grasping, the motion, the control, the safety, and the transfer — all seven — with the out-of-scope list’s exclusions stated. The dexterous hand would add the 2.3.4 module and would require the 4.1.1 §L.D.1’s in-list’s expansion, which is the scope’s decision, not the embodiment’s.

The evaluation’s resolution. The embodiment’s test episodes’ cost, times the metrics’ count, times the axes’ count, must fit the budget at the acceptance’s resolution. The running case’s per-episode cost is the 3.1.3 §L.E.1’s 25-second cycle, and the 400-episode test set’s 3-point resolution’s cost is 2.8 hours of cell time. The dexterous hand’s per-episode cost is the reorientation’s 1.2 seconds’ addition plus the multi-contact’s sim’s data, and the same resolution’s cost is the budget’s constraint.

The failure analysis’s observability. The embodiment’s failures must be observable with the sensors it has. The running case’s failures — the slip, the collision, the mis-grasp — are observable from the camera, the momentum observer, and the joint encoders. The dexterous hand’s failures — the internal-motion’s stall, the contact’s loss — are partly invisible without the tactile arrays, and the tactile arrays’ sim’s fidelity is the 3.4.1 §L.B.2’s hardest axis.

The criterion’s conclusion for the running case is that the parallel-jaw arm is the chosen embodiment. The dexterous hand is a future iteration’s in-list, and its choice’s criterion is the tactile’s data’s availability and the evaluation’s resolution’s budget, which the current capstone does not have.

Concept check. The dexterous hand’s in-hand reorientation is a capability the running case cannot demonstrate. Does that make the hand the better embodiment for the capstone?

Answer. No, by the decisiveness criterion: the capability is real, and the evaluation’s resolution within the budget is the constraint; the hand’s multi-contact’s metrics and the reorientation’s sub-tasks multiply the test set’s cost, and the acceptance’s resolution is worse. The capability is a future iteration’s in-list, not this capstone’s choice. [displaces M1]

Worked-example ladder.

WE-L.A.a (fully worked). The running case’s per-episode cost is 25 seconds. The dexterous hand’s is 25 seconds plus 1.2 seconds for the reorientation plus 10 seconds for the multi-contact’s simulator’s setup, so 36.2 seconds. Compute the test set’s cost for 400 episodes in each, and the resolution’s half-width at a 0.90 success rate for a 10-hour budget.

Solution. Running case: $400 \times 25 = 10{,}000$ seconds $= 2.78$ hours, giving a standard error of $\sqrt{0.9 \times 0.1/400} = 0.015$ and a half-width of 0.0295. Dexterous hand: $400 \times 36.2 = 14{,}480$ seconds $= 4.02$ hours; the same 10-hour budget gives $10 \times 3600/36.2 = 994$ episodes, a standard error of $\sqrt{0.09/994} = 0.0095$ and a half-width of 0.0187 — better, if the budget is 10 hours. The completion below makes the arithmetic’s nuance precise.

WE-L.A.b (completion). The budget is 3 hours, not 10. Recompute both resolutions.

Completion. Running case: $3 \times 3600/25 = 432$ episodes, half-width $1.96 \times \sqrt{0.09/432} = 0.0284$. Dexterous hand: $3 \times 3600/36.2 = 298$ episodes, half-width $1.96 \times \sqrt{0.09/298} = 0.0341$. At a 3-hour budget, the running case’s resolution is better; at a 10-hour budget, the hand’s is. The completion shows the choice depends on the budget’s size, and the criterion is the resolution at the budget, not an absolute ranking.

WE-L.A.c (faded). The running case’s six metrics are success, placement error, time, peak force, recovery, and safety violations. The dexterous hand would add the reorientation’s error and the internal-motion’s smoothness. Compute the metrics’ count’s effect on the test set’s size if each metric needs the same resolution.

Faded step. The size for one metric’s 3-point resolution is 384; the six-metric case’s conjunction’s failure probability requires the sizes to be considered jointly, not multiplied.

Completion. The metrics’ count’s effect is not a multiplication of the test set’s size; it is the acceptance’s conjunction’s failure’s probability’s rise, which is the 4.1.1 §L.C.2’s decision’s structure. The additional metrics multiply the decisions, not the episodes, and each decision’s threshold is the resolution’s requirement.

WE-L.A.d (contrastive near-miss). A team chooses the dexterous hand for its capability, and mid-project discovers the evaluation’s resolution is 0.05 at the acceptance’s threshold. Why it fails: the capability was chosen without the resolution’s arithmetic, and the evaluation’s acceptance is not decidable at the available budget. The fix: the decisiveness criterion, per §L.A.3, applied before the embodiment’s choice is frozen.

Comparative table: the alternatives’ costs.

Design principle P51.1. Choose the embodiment that makes the evaluation’s acceptance decidable within the budget; the capability’s ceiling is a future iteration’s in-list, not this capstone’s choice.

Consolidation checkpoint 1

(a) Three capabilities. You can name the four axes and the running case’s values. You can compare the alternatives on the axes. You can apply the decisiveness criterion.

(b) Transfer probe. The budget is 2 hours and the dexterous hand’s per-episode cost is 36 seconds. What resolution does the hand’s test set give at 0.90 success? Answer: $7200/36 = 200$ episodes, half-width $1.96 \times \sqrt{0.09/200} = 0.0416$, worse than the running case’s at the same budget.

(c) Remediation rule. If you cannot state why the cheaper embodiment’s evaluation is more precise at a fixed budget, re-read §L.A.2 before Part B.

Part B — The evaluation’s coupling

§L.B.1 — The metrics the embodiment supports

The evaluation’s metrics are not universal; each is a function of the embodiment’s sensors and mechanisms, and the coupling is the reason the embodiment’s choice is the evaluation’s design’s foundation.

Success rate is supported by every embodiment, with the achievement’s check’s definition’s dependence on the embodiment: the running case’s check is the cube’s final pose’s distance from the bin’s target pose; the dexterous hand’s check would include the reorientation’s target, which the parallel-jaw gripper cannot express.

Placement error is supported by every embodiment, but the tolerance depends on the gripper’s geometry and the object’s size. The running case’s 2 cm tolerance is set by the 5 cm cube’s half-width and the bin’s opening; the dexterous hand’s tolerance for the same cube is tighter, because the reorientation’s precision’s requirement is tighter.

Time is supported by every embodiment, and the bound depends on the embodiment’s kinematics: the 7-DOF arm’s extra joint’s reach’s advantage can shorten the transport’s time; the dexterous hand’s reorientation’s 1.2 seconds lengthens the time.

Peak contact force is supported by the running case’s momentum observer and by the dexterous hand’s tactile arrays, and its definition differs: the running case’s is the grasp’s normal force’s maximum, the hand’s is the fingertip’s force’s maximum, which is a different quantity with a different limit.

Recovery rate is supported by every embodiment, and its definition depends on the recovery’s mechanism’s availability: the running case’s recovery is the 3.3.2 §L.C.2’s state-changing retry; the dexterous hand’s is the regrasp’s gait cycle, which is a different kind of recovery.

Safety violations is supported by every embodiment, and its count depends on the safety stack’s layers’ inputs: the running case’s violations are the four-layer stack’s precondition’s failures; the dexterous hand’s would add the internal-motion’s constraints, and the mobile arm’s would add the base’s collision’s layer.

The metrics’ table’s dependence on the embodiment is the first form of the coupling. The second is the baselines.

§L.B.2 — The baselines the embodiment permits

The 4.1.1 §L.C.3’s precondition was that the evaluation states its baseline. The baseline’s availability depends on the embodiment, and the dependence is the second form of the coupling.

The running case’s baselines. The 1.1.4 specialist pipeline, the 3.1.4 fine-tuned VLA, and the 3.4.4 adapted policy. Each is available on the running arm, and each is a fully built system whose result is a number in a prior lesson. The baseline’s availability is the evaluation’s comparison’s content.

The dexterous hand’s baselines. The 2.3.4 Dactyl-style policy is the reference, and it is available as a published result on the Shadow hand, not on the running hand. The hand’s baseline would require a new training’s run, which is the capstone’s budget’s cost. The baseline’s unavailability is the evaluation’s comparison’s gap.

The 7-DOF arm’s baselines. The 1.1.4 pipeline is available with an IK’s null-space’s addition, which is the capstone’s work. The baseline’s modification’s cost is the integration’s, not the evaluation’s.

The mobile arm’s baselines. The 2.1.5 SLAM’s drift’s baseline is available, and the pick-and-place’s is the 1.1.4 pipeline on the base, which is the capstone’s work.

The baselines’ comparison’s table.

The baselines’ table’s role is the evaluation’s claim’s content: a capstone’s result is the margin over the baseline, and a baseline that requires a new training’s run is a cost that the budget’s resolution’s arithmetic must include.

§L.B.3 — The distributions the embodiment can reach

The 4.1.1 §L.C.3’s precondition was that the test set’s distribution is stated. The distribution’s reach depends on the embodiment, and the dependence is the third form of the coupling.

The running case’s reachable distribution. The table’s workspace, the cube’s 5 cm geometry, the clutter’s three objects, and the bin’s opening. The reachable distribution’s axes are the 4.1.1 §L.C.2’s four: seen task, unseen object, unseen phrasing, unseen clutter.

The dexterous hand’s reachable distribution. The table’s workspace plus the reorientation’s target’s orientation’s range. The hand’s distribution includes the in-hand’s motions, and the test set’s sample must cover the orientation’s range, which is a new axis.

The mobile arm’s reachable distribution. The workspace’s multiple tables, which the base’s motion reaches, and the localization’s drift’s conditions. The mobile arm’s distribution’s axes include the site’s axis, which the running case does not have.

The 7-DOF arm’s reachable distribution. The running case’s distribution plus the redundancy’s use’s conditions — the joint-limit’s avoidance’s scenes and the obstacle’s clearance’s scenes — which are the new axes.

The distributions’ comparison’s table.

The distributions’ table’s role is the test set’s size’s arithmetic: each axis multiplies the test set’s episodes, and the new axis’s data’s cost is the budget’s constraint.

Self-explanation 2. Why is the distribution’s reach a coupling rather than a free choice?

Answer. Because the test set’s distribution is the embodiment’s reachable distribution’s sample; the test episodes are real executions, and an embodiment cannot be tested on a distribution it cannot reach. The sample’s reach is the embodiment’s, and the choice’s coupling is the reach’s function of the embodiment.

Concept check. A capstone’s test set includes an axis for “the cube’s orientation’s range” but the gripper is a parallel-jaw that can only grasp at two orientations. Does the axis make sense?

Answer. No; the axis’s episodes cannot be executed on the embodiment, and the test set’s distribution’s axis is unreachable. The mechanism is the embodiment’s reach’s coupling, and the fix is either the axis’s removal or the embodiment’s change. [displaces M2]

Worked-example ladder.

WE-L.B.a (fully worked). The running case’s test set has four axes and 400 episodes. The dexterous hand’s would have five axes and, at the same per-axis’s size, 500 episodes. Compute the total cost’s difference at 25 seconds and 36.2 seconds per episode, and state the resolution’s difference at 3 hours.

Solution. Running case: $400 \times 25 = 10{,}000$ seconds = 2.78 hours, within the 3-hour budget, half-width $1.96 \times \sqrt{0.09/400} = 0.0295$. Dexterous hand: $500 \times 36.2 = 18{,}100$ seconds = 5.03 hours, above the 3-hour budget; the available budget’s 298 episodes give 5 axes × 60 episodes per axis, and the per-axis’s half-width is $1.96 \times \sqrt{0.09/60} = 0.076$, much worse than the running case’s 0.0295. The cost’s difference is the resolution’s difference.

WE-L.B.b (completion). The budget rises to 6 hours. Recompute both.

Completion. Running case: $6 \times 3600/25 = 864$ episodes, per-axis 216, half-width $1.96 \times \sqrt{0.09/216} = 0.040$. Dexterous hand: $6 \times 3600/36.2 = 596$ episodes, per-axis 119, half-width $1.96 \times \sqrt{0.09/119} = 0.054$. The running case’s resolution is still better, and the hand’s five axes’ cost is the gap’s source. The completion shows the axes’ count’s effect on the resolution.

WE-L.B.c (faded). Write the metrics-baseline matrix for the running case and the dexterous hand, with each cell stating the metric’s availability.

Faded step. Running case: success, placement error, time, peak force, recovery, safety violations, all available; baselines: three prior systems. Dexterous hand: the six plus the reorientation’s error and the internal-motion’s smoothness, all available in principle; baselines: one published result, requiring a new training’s run.

Completion. The matrix’s role is the evaluation’s claim’s content; the dexterous hand’s availability is theoretical, and the baseline’s cost is the budget’s constraint.

WE-L.B.d (contrastive near-miss). A team uses the running case’s four axes for the dexterous hand’s test set, and the hand’s reorientation’s performance is not measured. Why it fails: the test set’s distribution’s axes are the running case’s reach’s, not the hand’s; the reorientation is a capability without a test. The fix: the axes’ expansion to the hand’s reach’s, which is the budget’s cost.

Comparative table: the coupling’s three forms.

Design principle P51.2. Choose the metrics, the baselines, and the distributions together with the embodiment; each is a function of the embodiment, and the coupling is the evaluation’s design.

Consolidation checkpoint 2

(a) Three capabilities. You can name the metrics the embodiment supports and the new ones it would add. You can state the baseline’s role and the cost of an unavailable baseline. You can state the distribution’s reach’s coupling.

(b) Transfer probe. A 7-DOF arm’s redundancy adds one axis. At 400 episodes per axis and 30 seconds per episode, what is the test set’s cost and the per-axis’s half-width at a 0.90 success rate? Answer: $5 \times 400 = 2000$ episodes, $2000 \times 30 = 16.7$ hours, per-axis 400, half-width $1.96 \times \sqrt{0.09/400} = 0.0295$.

(c) Remediation rule. If you cannot state why the baseline’s availability is a coupling, re-read §L.B.2 before Part C.

Part C — The held-out design and the sample size

§L.C.1 — The four-plus-one axes

The 4.1.1 §L.C.2’s four axes were seen task, unseen object, unseen phrasing, and unseen clutter. The capstone adds a fifth axis: unseen embodiment’s condition, which is the distribution’s shift along the embodiment’s axes — the camera’s pose’s range, the object’s position’s range within the reachable workspace, and the clutter’s configurations’ new instances.

The fifth axis’s purpose is the generalization’s measurement within the chosen embodiment, not across embodiments. The capstone’s claim is about the running case, and the claim’s scope is the running case’s distribution; the fifth axis measures the model’s generalization to the distribution’s unseen region, not to another embodiment’s.

The fifth axis’s episodes are the running case’s, with the conditions’ variation. The axis’s stratification is by the camera’s pose’s bin, the object’s position’s bin, and the clutter’s configuration’s instance. The stratification’s bins are the 3.1.3 §L.D.2’s collection’s batches’ complements.

The five axes’ table.

The axes’ table’s role is the report’s structure: each axis’s row is the claim’s scope, and the aggregate is the headline’s summary.

§L.C.2 — The metrics’ decisions

The 4.1.1 §L.C.1’s six metrics are the primary set, and the capstone’s fifth axis does not add metrics — it adds the conditions under which the six are measured.

The metrics’ decisions’ structure is the 4.1.1 §L.E.2’s conjunction: each metric’s bound is a necessary condition, and the acceptance fails if any is missed. The conjunction’s failure’s probability’s rise with the metrics’ count is the reason the metrics’ set is chosen, not enumerated.

The decision’s threshold is the bound’s, and the decision’s resolution is the test set’s size’s. A metric with a 3-point resolution and a 0.90 bound requires 384 episodes; a metric with a 1-point resolution requires $1.96^2 \times 0.09/0.01^2 = 3457$ episodes, nine times more. The resolution’s requirement is the metric’s choice’s cost, and the metrics’ set’s size is the sum of the decisions’ resolutions.

The running case’s six metrics’ decisions.

The table’s role is the arithmetic’s total: the episodes’ count is the maximum, not the sum, because the same episode supports all six metrics. The test set’s size is 400, which is the maximum of the six decisions’ resolutions.

§L.C.3 — The sample size and the budget’s constraint

The test set’s size is the maximum of the metrics’ decisions’ required sizes, not the sum and not the product. The same episode is measured by all six metrics, and the acceptance’s conjunction’s failure’s probability is the joint distribution’s, not the product of the marginal decisions’. The 4.1.1 §L.C.3’s three preconditions are the joint distribution’s statement’s form.

The size’s arithmetic’s structure is the resolution’s requirement’s binding constraint. The running case’s 400 episodes are the placement error’s and the safety’s decisions’ maximum, and the success’s 384 is the second. The 400 is the test set’s size, and its cost is the budget’s.

The budget’s constraint is the resolution’s, not the size’s. The budget’s 3 hours gives $3 \times 3600/25 = 432$ episodes, which supports the 400’s maximum with margin. If the budget fell to 2 hours, the episodes’ 288 would give a 3.5-point resolution, which is the acceptance’s resolution’s loss, not the size’s. The choice is the resolution’s, and the budget’s constraint is the resolution’s.

The stratification’s allocation is the axes’ counts. The 400 episodes are allocated across the five axes proportionally to the axes’ importance: the seen task’s 100, the unseen object’s 100, the unseen phrasing’s 80, the unseen clutter’s 80, and the unseen condition’s 40. The axes’ allocation’s sums are the test set’s size, and the per-axis’s half-width is the allocation’s resolution.

The allocation’s design is the 3.1.3 §L.D.3’s balancing’s structure: the axes’ importance is the claim’s content, and the capstone’s claim is the running case’s performance across the four-plus-one’s conditions.

Self-explanation 3. Why is the sample size the maximum rather than the sum of the decisions’ resolutions?

Answer. Because the same episode is measured by every metric, so the episode count’s requirement is the largest metric’s requirement, not the sum; the sum would be the requirement if each metric needed its own episodes, and the joint measurement is the capstone’s economy.

Concept check. A metric with a 0.5-point resolution requires 15,000 episodes. Is that within the 3-hour budget at 25 seconds per episode?

Answer. No; 15,000 episodes at 25 seconds is 104 hours, far above the 3-hour budget. The metric’s resolution’s requirement is the budget’s constraint, and the choice is either to relax the resolution to a level the budget affords or to drop the metric. [displaces M3]

Worked-example ladder.

WE-L.C.a (fully worked). The five axes’ allocations are 100, 100, 80, 80, 40. Compute each axis’s half-width at a 0.90 success rate and the acceptance’s weakest axis.

Solution. The seen task’s 100: $1.96 \times \sqrt{0.09/100} = 0.0588$. The unseen object’s 100: 0.0588. The phrasing’s 80: $1.96 \times \sqrt{0.09/80} = 0.0657$. The clutter’s 80: 0.0657. The condition’s 40: $1.96 \times \sqrt{0.09/40} = 0.0930$. The weakest axis is the unseen condition’s, and its 0.093 is the claim’s resolution’s limit.

WE-L.C.b (completion). The unseen condition’s axis is raised to 80 episodes by reducing the seen task’s to 60. Recompute both and state whether the change is an improvement.

Completion. Seen task’s 60: $1.96 \times \sqrt{0.09/60} = 0.0759$. Unseen condition’s 80: 0.0657. The improvement is the condition’s 0.093 to 0.066, and the cost is the seen’s 0.059 to 0.076. The change is an improvement if the condition’s axis is the claim’s content, and the choice is by the claim’s emphasis.

WE-L.C.c (faded). Write the test set’s size’s arithmetic for a 1-point resolution’s success rate and state whether the 3-hour budget affords it.

Faded step. The size is $1.96^2 \times 0.09/0.01^2 = 3457$ episodes; the cost is $3457 \times 25 = 86{,}425$ seconds = 24 hours, far above the 3-hour budget.

Completion. The 3-hour budget does not afford the 1-point resolution, and the choice is the 3-point resolution’s acceptance’s claim’s scope.

WE-L.C.d (contrastive near-miss). A team’s test set has 400 episodes allocated uniformly across five axes, so 80 each. The unseen condition’s axis is the claim’s key content, and its 80’s half-width is 0.0657, which is the claim’s resolution’s limit. Why it fails: the allocation was uniform rather than proportional to the axes’ importance, and the key axis’s resolution is worse than it could be. The fix: the allocation’s weighting, per §L.C.3.

Comparative table: the axes and the allocations.

Design principle P51.3. Choose the test set’s size as the maximum of the metrics’ decisions’ resolutions, allocate the axes by the claim’s emphasis, and state the budget’s constraint as the resolution’s, not the size’s.

Consolidation checkpoint 3

(a) Three capabilities. You can name the five axes and their stratification. You can compute the sample size from a resolution’s requirement. You can allocate the axes and compute each axis’s half-width.

(b) Transfer probe. A metric with a 2-point resolution requires 865 episodes. At 25 seconds per episode, what is the cost, and does the 3-hour budget afford it? Answer: $865 \times 25 = 21{,}625$ seconds = 6 hours, above the 3-hour budget; the choice is the 3.5-point resolution at 288 episodes or a larger budget.

(c) Remediation rule. If you cannot state why the size is the maximum and not the sum, re-read §L.C.3 before Part D.

Part D — The decisiveness criterion and the risk register

§L.D.1 — The commitment’s cost

The embodiment’s and the evaluation’s choices are a commitment: once the test set’s distribution, the baselines, and the metrics are frozen, changing any of them invalidates the held-out suite’s evidence and the metrics’ history’s comparison.

The commitment’s cost has three parts.

The held-out suite’s invalidation. The test episodes were selected for the chosen embodiment’s distribution; a different embodiment’s distribution is a different sample, and the frozen test set’s episodes are not the new distribution’s sample.

The baselines’ invalidation. The baselines’ numbers were measured on the chosen embodiment; a different embodiment’s baseline is a new training’s run, which is the budget’s cost.

The metrics’ history’s invalidation. The metrics’ values’ time series is the 4.3 lesson’s iteration’s evidence; a metric’s redefinition breaks the time series, and the iteration’s progress is unmeasurable.

The commitment’s cost is the reason the choice is made before the build, in the 3.1.4 §L.A.1’s Phase 1’s baseline audit’s start. The choice’s timing is the cost’s minimizer: a choice made early costs a day’s analysis; a choice made late costs the held-out suite’s redone episodes and the metrics’ history’s reset.

Self-explanation 1. Why does the choice’s timing minimize the cost?

Answer. Because the cost’s three parts are the held-out suite, the baselines, and the metrics’ history, and each is proportional to the work done before the change; a choice made before the work is done has no work to invalidate.

§L.D.2 — What would change my mind

The “what would change my mind” test is the choice’s falsifiability: for each of the choice’s components, the test states the observation that would make the choice wrong.

The running case’s tests.

The embodiment. The choice would be wrong if the evaluation’s resolution at the budget were worse than an alternative’s, or if the failure analysis’s observability were insufficient — e.g., if the slip’s detection’s latency’s requirement exceeded the momentum observer’s 3 ms.

The metrics. The choice would be wrong if a metric’s decision’s resolution were unaffordable and the metric were the claim’s key content — e.g., if the placement error’s 0.5 cm resolution required more than the budget’s episodes.

The baselines. The choice would be wrong if the baseline’s number were not comparable — e.g., if the 1.1.4 specialist’s number were measured on a different test set’s distribution.

The axes. The choice would be wrong if an axis’s claim were not the capstone’s content — e.g., if the unseen phrasing’s axis were uninformative because the deployment’s operators use only the seen phrasings.

The test’s role is the choice’s justification: the choice is not the “best” in an absolute sense but the choice whose failures are stated and whose observations would falsify it. The 4.4.3 lesson’s “communicating the unknown unknowns” uses the test as its structure.

§L.D.3 — The risk register’s update

The 4.1.1 §L.D.3’s risk register had five entries. The embodiment and the evaluation’s choice adds four, and the additions are the choice’s risks.

The register’s entries are the 3.3.4 §L.E.3’s five fields, and their residuals are the acceptance’s residuals.

The resolution’s gap is the risk that the acceptance’s resolution is too coarse to distinguish the capstone’s result from the baseline’s; the mitigation is the size’s arithmetic, and the residual is the budget’s ceiling.

The baseline’s incomparability is the risk that the baseline’s number was measured on a different distribution; the mitigation is the distribution’s statement, and the residual is the comparison’s scope — the claim is about the shared distribution, not the whole.

The axis’s uninformative is the risk that an axis’s claim is not the capstone’s content; the mitigation is the allocation’s weighting, and the residual is the claim’s emphasis’s choice.

The fifth axis’s data’s cost is the risk that the unseen condition’s axis’s data exceeds the budget; the mitigation is the allocation’s plan, and the residual is the unseen conditions the axis’s sample does not cover.

Self-explanation 2. Why is the resolution’s gap a residual rather than a mitigation’s complete coverage?

Answer. Because the budget’s ceiling is finite, and the resolution’s requirement’s satisfaction is the budget’s function; a mitigation that could always achieve any resolution would be a claim of unlimited budget, which is not the deployment’s reality.

Concept check. A capstone’s resolution is 0.06 at the acceptance’s budget, and the baseline’s difference is 0.04. Is the acceptance decidable?

Answer. No; the resolution’s 0.06 is larger than the difference’s 0.04, and the acceptance’s comparison cannot distinguish the capstone from the baseline. The mechanism is the resolution’s gap’s risk, and the fix is a larger budget or a narrower baseline’s gap. [displaces M4]

Worked-example ladder.

WE-L.D.a (fully worked). The running case’s success rate’s difference over the 3.1.4 baseline is 0.04, and the test set’s half-width is 0.0295. Is the difference significant?

Solution. The difference’s standard error is $\sqrt{0.09/400 + 0.09/400} = 0.0212$, and the 95% interval is $[0.04 - 0.0416, 0.04 + 0.0416] = [-0.0016, 0.0816]$, which includes zero. The difference is not significant at the 400-episode test set, and the resolution’s gap’s risk is realized.

WE-L.D.b (completion). The test set is expanded to 1,600. Recompute and decide. Completion. The standard error is $\sqrt{0.09/1600 + 0.09/1600} = 0.0106$, and the interval is $[0.019, 0.061]$, which excludes zero. The difference is significant, and the resolution’s gap is closed.

WE-L.D.c (faded). Write the “what would change my mind” test for the baseline’s choice.

Faded step. The choice would be wrong if the baseline’s number were measured on a different test set’s distribution; the test’s observation is the distribution’s comparison.

Completion. The test’s observation is the distribution’s statement’s comparison, and the falsifiability is the claim’s scope’s statement.

WE-L.D.d (contrastive near-miss). A team’s resolution is 0.03 and the baseline’s difference is 0.02, so the interval includes zero; the team concludes the capstone has no benefit. Why it fails: the conclusion is the resolution’s, not the system’s; the capstone’s difference could be real and the resolution’s gap’s risk is the evidence’s limit. The fix: the resolution’s gap’s risk’s mitigation — a larger test set — and the claim’s scope’s statement that the difference is not resolvable at the current budget.

Comparative table: the register’s additions.

Design principle P51.4. Choose with a falsifiable statement and a register; the choice’s justification is the observation that would falsify it, and the register’s residuals are the choice’s acceptance.

Consolidation checkpoint 4

(a) Three capabilities. You can state the commitment’s three costs. You can write the “what would change my mind” test for each component. You can compute the difference’s significance from the test set’s size.

(b) Transfer probe. The difference is 0.05 and the test set’s standard error is 0.03. Is it significant? Answer: the interval is $[0.05 - 0.0588, 0.05 + 0.0588] = [-0.0088, 0.1088]$, which includes zero; the difference is not significant at this test set’s size.

(c) Remediation rule. If you cannot state why the commitment’s timing minimizes its cost, re-read §L.D.1 before Part E.

Part E — The running case

§L.E.1 — The choice

The running case’s choice is the parallel-jaw arm, the eye-in-hand camera, the existing gripper, and the four-plus-one axes.

The embodiment. The 6-DOF arm with alternating $z$/$y$ axes, equal links $a = 0.2$ m, the parallel-jaw gripper with 0–0.08 m opening and 5 N standard grip, and the eye-in-hand RGB-D camera. The choice is justified by the decisiveness criterion: the evaluation’s resolution at the 3-hour budget is 0.0295 on the success’s axis and 0.5 cm on the placement error’s, and the failure analysis’s observability is the momentum observer’s 3 ms latency and the camera’s pose.

The metrics. The six of 4.1.1 §L.C.1, with the bounds’ table of §L.C.2. The decisions’ resolutions’ maximum is the placement error’s 400 episodes, which is the test set’s size.

The baselines. The 1.1.4 specialist pipeline, the 3.1.4 fine-tuned VLA, and the 3.4.4 adapted policy. Each is available on the running arm, and each is a fully built system whose number is a prior lesson’s result.

The axes. The 4.1.1 §L.C.2’s four plus the fifth of §L.C.1, with the allocations of §L.C.3’s table.

The held-out set’s cost. The 400 episodes’ 2.78 hours, within the 3-hour budget with a 0.22-hour margin.

§L.E.2 — The frozen evaluation

The evaluation’s frozen design.

The frozen design is the capstone’s contract with its evaluation, and the freeze’s timestamp is the 3.1.4 §L.A.1’s Phase 1’s baseline audit’s start.

§L.E.3 — The artefacts

The choice’s artefacts are the 4.1.1 §L.E.3’s six plus the choice’s three.

The 4.1.1 six. The capability matrix, the interface contracts, the metrics’ table, the failure taxonomy, the risk register, and the code.

The choice’s three. The embodiment’s comparison’s table, the evaluation’s frozen design’s document, and the “what would change my mind” test’s list.

The nine artefacts’ completeness is the 4.1.3 lesson’s held-out suite’s design’s input, and the 4.4.1 lesson’s defense’s evidence.

Self-explanation 3. Why is the “what would change my mind” list an artefact rather than a section of the report?

Answer. Because the list’s items are the choice’s falsifiability’s statements, and their evidence is the observations’ specification; the list’s structured form is the defense’s evidence, and the prose’s narrative is the defense’s complement, not the evidence.

Worked-example ladder.

WE-L.E.a (fully worked). The running case’s test set’s 400 episodes at 25 seconds are 2.78 hours. The budget is 3 hours. Compute the margin and state whether the fifth axis’s 40 episodes’ allocation can be doubled to 80.

Solution. The margin is 0.22 hours $= 792$ seconds, which is 31.7 episodes. Doubling the fifth axis’s 40 to 80 adds 40 episodes, which is 1000 seconds, above the margin. The doubling is not affordable at the 3-hour budget, and the fifth axis’s 40 remains. The completion below shows the alternative.

WE-L.E.b (completion). The fifth axis’s 40 are raised to 60 by reducing the seen task’s 100 to 80. Recompute the total and the half-widths.

Completion. The total is 400 (unchanged), and the seen task’s 80: $1.96 \times \sqrt{0.09/80} = 0.0657$; the fifth axis’s 60: $1.96 \times \sqrt{0.09/60} = 0.0759$. The seen task’s resolution’s cost is the seen’s 0.059 to 0.066, and the fifth’s improvement is 0.093 to 0.076. The change is an improvement if the fifth’s axis is the claim’s key content.

WE-L.E.c (faded). Write the “what would change my mind” test for the metrics’ choice.

Faded step. The choice would be wrong if a metric’s decision’s resolution were unaffordable and the metric were the claim’s key content; the test’s observation is the metric’s resolution’s requirement against the budget’s episodes.

Completion. The test’s observation is the placement error’s 0.5 cm resolution’s 400 episodes, which is the budget’s maximum; the metric’s choice’s falsifiability is the resolution’s affordability’s statement.

WE-L.E.d (contrastive near-miss). A team’s fifth axis’s 40 episodes give a 0.093 half-width, and the claim’s key content is the fifth axis. The team’s report’s conclusion is “the model generalizes to the unseen conditions” without stating the resolution. Why it fails: the claim’s scope is the 0.093’s resolution, and the “generalizes” is a statement the 40 episodes cannot support. The fix: the claim’s scope’s statement and the fifth axis’s expansion’s plan.

Comparative table: the choice’s artefacts.

Design principle P51.5. Freeze the choice with its artefacts and its falsifiability’s list; the choice’s justification is the observation that would falsify it, and the artefacts are the defense’s evidence.

Consolidation checkpoint 5

(a) Three capabilities. You can state the running case’s choice and its justification. You can apply the frozen design’s components. You can name the choice’s three additional artefacts.

(b) Transfer probe. The budget rises to 4 hours. What does the fifth axis’s allocation become? Answer: the budget’s 576 episodes at 25 seconds, and the fifth axis’s share rises proportionally with the other axes’ if the allocation’s weighting is preserved; the choice is the allocation’s weighting’s decision.

(c) Remediation rule. If you cannot state why the fifth axis’s 40 is the claim’s resolution’s limit, redo WE-L.E.a before the lab.

Failure taxonomy

The table covers the three lab corruptions plus three failures not exercised in the lab: the baseline’s mismatch, the axis’s uninformative, and the creep’s return.

The lab

Task. Build the embodiment’s comparison’s table, the evaluation’s frozen design, and the “what would change my mind” list for the running case and the three alternatives, and run the decisiveness criterion’s arithmetic at three budgets.

Conditions. Python 3.10+, NumPy. The costs and metrics’ bounds are the lessons’ prior results; the arithmetic is the paper evaluation’s. Seed all RNGs.

Controls. (i) The running case at a 3-hour budget; (ii) the running case at a 6-hour budget; (iii) the dexterous hand at a 3-hour budget; (iv) the 7-DOF arm at a 3-hour budget.

Metrics.

Pre-registered hypotheses.

H1. The running case at a 3-hour budget has a half-width below 0.03, and the acceptance’s difference over the baseline is significant at the 0.04 difference.

H2. The dexterous hand at a 3-hour budget has a half-width above 0.035, because its per-episode cost and its five axes’ allocation are higher.

H3. The 7-DOF arm’s additional axis’s cost reduces its resolution’s advantage, and its half-width is between the running case’s and the hand’s.

H4. The 6-hour budget’s running case has a half-width below 0.025, and the fifth axis’s allocation is the choice’s binding constraint.

Break-it sequence.

1. Use the sum of the metrics’ resolutions silently. The test set’s size is nine times the maximum. Silent: the arithmetic looks conservative.
2. Allocate the axes uniformly silently. The key axis’s resolution is worse than the weighting’s. Silent: the allocations sum to the total.
3. Measure the baseline on a different distribution silently. The comparison’s difference is the distribution’s, not the system’s. Silent: the numbers look plausible.

Acceptance criteria.

What good looks like. A good artefact has five properties. (1) The comparison’s table’s cells are the axes’ values, not a ranking. (2) The frozen design’s components are each a document, not a table’s assertion. (3) The “what would change my mind” list’s items are observations, not opinions. (4) The break-it results are a table. (5) The register’s nine entries have residuals.

A runnable core of the decisiveness criterion’s arithmetic, seeded and annotated:

# lab_4_1_2.py — embodiment and evaluation's decisiveness
import numpy as np
np.random.seed(0)

# Per-embodiment costs and axes' counts (from the lessons' results)
EMBODIMENTS = {
    "running_case":  {"cost_s": 25.0, "axes": 5, "metrics": 6},
    "7dof_arm":      {"cost_s": 28.0, "axes": 5, "metrics": 6},
    "dexterous_hand":{"cost_s": 36.2, "axes": 5, "metrics": 8},
    "mobile_arm":    {"cost_s": 30.0, "axes": 5, "metrics": 7},
}

def resolution(budget_h, cost_s, axes, $n_ep_per_axis=80$, p=0.90):
    total = int(budget_h * 3600 / cost_s)
    per_axis = total // axes
    half = 1.96 * np.sqrt(p * (1 - p) / max(per_axis, 1))
    return total, per_axis, half

for name, spec in EMBODIMENTS.items():
    for budget in (3, 6):
        total, per_axis, half = resolution(
            budget, spec["cost_s"], spec["axes"])
        print(f"{name:16s} budget={budget}h  total={total:4d}  "
              f"per_axis={per_axis:3d}  half={half:.3f}")

# Difference's significance at the running case's 400 episodes
p, n = 0.90, 400
se = np.sqrt(p*(1-p)/n + p*(1-p)/n)
diff = 0.04
lo, hi = diff - 1.96*se, diff + 1.96*se
print(f"difference interval: [{lo:.4f}, {hi:.4f}]  (significance: {lo > 0})")

# Metrics' decisions' resolutions
decisions = {
    "success":    {"bound": 0.90, "res": 0.03, "size": 384},
    "placement":  {"bound": 0.02, "res": 0.005, "size": 400},
    "time":       {"bound": 10.0, "res": 1.0,  "size": 100},
    "peak_force": {"bound": 20.0, "res": 2.0,  "size": 100},
    "recovery":   {"bound": 0.50, "res": 0.10, "size": 100},
    "safety":     {"bound": 0.0,  "res": 0.0,  "size": 400},
}
max_size = max(d["size"] for d in decisions.values())
print(f"test set's size = {max_size}  (max, not sum)")

Line 5–10 define the embodiments’ per-episode costs, axes’ counts, and metrics’ counts. Lines 12–18 compute the total episodes, the per-axis allocation, and the half-width at two budgets. Lines 20–24 compute the difference’s interval and its significance. Lines 26–35 define the metrics’ decisions and compute the test set’s size as the maximum, not the sum.

Cheat sheet

Exercises with solutions

E 4.1.2.1. Name the four embodiment axes and give the running case’s value on each.

S 4.1.2.1. DOF: 6, exact. Contact: two-point, parallel-jaw, 0-0.08 m. Sensors: eye-in-hand RGB-D, joint encoders, momentum observer. Sim fidelity: moderate, $k_c = 10^5$ N/m. The lesson: the axes are the comparison’s decomposition, and the running case is the cheapest on each.

E 4.1.2.2 (transfer). A dexterous hand’s per-episode cost is 36.2 seconds, and the budget is 2 hours. Compute the episodes and the per-axis’s half-width at five axes.

S 4.1.2.2. $7200/36.2 = 199$ episodes, per-axis 39, half-width $1.96 \times \sqrt{0.09/39} = 0.0941$. The lesson: the per-episode cost’s 45% increase and the five axes’ allocation give a resolution more than twice as wide as the running case’s.

E 4.1.2.3. State why the evaluation’s choice is coupled to the embodiment’s through the three forms.

S 4.1.2.3. The metrics an embodiment supports, the baselines available, and the distributions the test set can sample are all functions of the embodiment. The lesson: the three forms are the coupling’s structure, and the choice is the evaluation’s design’s foundation.

E 4.1.2.4 (diagnostic). A capstone’s test set has an axis for the cube’s orientation’s range but the parallel-jaw gripper can only grasp at two orientations. Which coupling’s form fails?

S 4.1.2.4. The distributions’ reach; the axis’s episodes cannot be executed on the embodiment. The fix is the axis’s removal or the embodiment’s change. The lesson: the distribution’s reach is the embodiment’s, not the design’s free choice.

E 4.1.2.5. Compute the test set’s size for a 1-point resolution at a 0.90 success rate, and state the cost at 25 seconds per episode.

S 4.1.2.5. $n = 1.96^2 \times 0.09/0.01^2 = 3457$; $3457 \times 25 = 86{,}425$ seconds = 24 hours. The lesson: the 1-point resolution’s cost is far above the 3-hour budget, and the choice is the 3-point’s scope.

E 4.1.2.6 (transfer). The fifth axis’s 40 episodes give a 0.093 half-width. The claim’s key content is the fifth axis. What is the fix and its cost?

S 4.1.2.6. The fifth axis’s allocation’s increase to 80 episodes, at the seen task’s reduction to 60 or the budget’s rise to 3.2 hours. The lesson: the allocation’s weighting is the claim’s emphasis’s expression.

E 4.1.2.7 (diagnostic). The difference over the baseline is 0.04 and the test set’s half-width is 0.03. Is the acceptance decidable?

S 4.1.2.7. The difference’s interval is $[0.04 - 0.0588, 0.04 + 0.0588]$, which includes zero; the acceptance is not decidable at this resolution. The fix is a larger test set or a narrower baseline’s gap. The lesson: the resolution’s gap’s risk is realized when the interval includes zero.

E 4.1.2.8. Write the “what would change my mind” test for the embodiment’s choice.

S 4.1.2.8. The observation is the evaluation’s resolution’s arithmetic at the budget and the failure analysis’s observability’s test; if an alternative’s resolution were better or the observability were insufficient, the choice would be wrong. The lesson: the choice’s justification is the observation that would falsify it.

E 4.1.2.9 (transfer). The register’s resolution’s gap’s residual is the budget’s ceiling. What does the residual’s acceptance’s statement mean?

S 4.1.2.9. The operator accepts that the claim’s scope is the resolution the budget affords, and the residual is the gap between that resolution and the deployment’s need. The lesson: the residual is what the operator accepts, not what the mitigation achieves.

Exit test

You are ready for Lesson 4.1.3 when you can, without notes:

1. Name the four embodiment axes and the running case’s values.
2. Compare the running case against the three alternatives on the axes.
3. Apply the decisiveness criterion to a choice.
4. State the coupling’s three forms and one example of each.
5. Build the metrics-baseline matrix.
6. State the baseline’s role and the cost of an unavailable baseline.
7. Name the five axes and their stratification.
8. Compute the test set’s size as the maximum, not the sum.
9. Allocate the axes and compute each axis’s half-width.
10. State the commitment’s three costs and the timing’s minimization.
11. Write the “what would change my mind” test for each component.
12. Compute the difference’s significance from the test set’s size.
13. Write the register’s four additional entries with residuals.
14. Freeze the running case’s choice and its artifacts.
15. State why the fifth axis’s 40 is the claim’s resolution’s limit.

Cross-references. 4.1.1 supplies the capstone’s definition, the six metrics, the four axes, and the register restated in §L.C.1 and §L.D.3. 2.3.4 supplies the dexterity count restated in §L.A.1. 3.4.1 supplies the four divergence axes restated in §L.A.1. 3.3.4 supplies the register restated in §L.D.3. 3.1.4 supplies the decision conjunction and the freeze restated in §L.D.1 and §L.E.2. 1.1.1 supplies the running arm’s geometry restated in §L.A.2. Lesson 4.1.3 designs the held-out suite’s stratification against this design. Module 4.2 builds the seven subsystems on the chosen embodiment.

Ready. The next lesson to write is 4.1.3 — Evaluation Design and Held-Out Test Suites. Say “next” to generate it.