Lesson 4.4.2 — Proposing Concrete Improvements

Week 42

Prerequisite lessons: 4.4.1 (architecting the defense), 4.3.3 (long-horizon robustness), 4.3.2 (diagnosing sim-to-real failures), 4.3.1 (systematic experimentation and ablations), 4.2.4 (safety and recovery), 3.4.4 (real-to-sim adaptation), 3.1.4 (VLA fine-tuning).

Prior results used in this lesson. From 4.4.1: the vulnerability map, the residuals’ weights, the claim-evidence-warrant table, and the “what would change my mind” statement, restated in §L.A.1 and §L.D.1. From 4.3.3: the horizon’s failure law, the maintenance’s schedule, and the extrapolation’s contract, restated in §L.B.2 and §L.E.2. From 4.3.2: the diagnosis’s stopping rule and the evidence’s chain, restated in §L.B.3. From 4.3.1: the registry’s six fields, the power’s arithmetic, and the CI’s lower bound, restated in §L.C.1. From 4.2.4: the register’s residuals and the recovery’s five kinds, restated in §L.D.3. From 3.4.4: the adaptation’s loop and the mapping’s drift, restated in §L.E.2. From 3.1.4: the decision conjunction and the rollback, restated in §L.D.2. Each is restated at the point of use.

What this lesson hands forward. The proposal’s five fields, the value-cost frontier, the improvement’s protocol, and the improvement’s registry. Lesson 4.4.3 (communicating the unknown unknowns) uses the improvement registry’s residuals as the unknown-unknowns artefact’s forward-looking section. The curriculum’s exit test is the defense’s passing, and the proposals are the defense’s strongest signal of mastery.

The running case in one sentence. The running arm, gripper, eye-in-hand camera, table, cube, bin, divider, and three distractors are unchanged; 4.4.1 §L.B.3 ranks the vulnerability map’s residuals, and its top three are the composed recovery rate below the floor requirement (weight 25), the H4 register row whose I5 detection margin is negative (weight 15), and the recovery-effectiveness decay inside the reset interval (weight 12), and this lesson asks how to turn each residual into a concrete proposal — a specific change, an expected effect, a cost, a measurement, and a kill criterion — that the committee can evaluate and that the arithmetic can refute.

§L.0 — Lesson map, objectives, and misconceptions

Observable outcomes. By the end of this lesson you can, without notes: (1) state the difference between a wish and a proposal and write the proposal’s five fields; (2) compute the expected value from the residual’s weight, the delta, and the probability of success; (3) compute the value-cost ratio and rank the portfolio; (4) identify the dependencies between improvements and order them topologically; (5) write the improvement protocol’s three parts — the change, the invariants, the kill criterion; (6) state the improvement registry’s eight fields; (7) design the improvement’s experiment with the pre-registered paired design and the CI’s lower bound; (8) state the acceptance’s three conditions and the rollback’s criteria; (9) name the four failure modes and their signatures; (10) assemble the running case’s portfolio of the top three proposals with their expected values, costs, and the deployment plan.

Misconception inventory.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

Q1. In 4.4.1 §L.B.3, the vulnerability map’s top three residuals are the composed recovery rate below the floor requirement (weight 25), the H4 I5 detection margin (weight 15), and the recovery-effectiveness decay (weight 12). What does the weight represent, and what does the improvement’s expected value do to it?

Answer. The weight is the probability times the consequence — the residual’s vulnerability magnitude, on the 1–25 scale 4.4.1 builds from two 1–5 ratings. The improvement’s expected value is the weight’s reduction that the improvement’s success would produce, discounted by the probability that the improvement succeeds. Used in §L.B.1 to define the expected value.

Q2. In 4.3.1 §L.C.1, the power arithmetic’s MDE at 400 episodes was 5.8 points. What does the MDE imply for an improvement whose expected effect is below it?

Answer. It implies the effect is not detectable at the current suite’s size, and the measurement requires a larger suite or a paired design that cancels the shared variance. The measurement field must state the detectable threshold, and the honest proposal states that its effect sits below it. Used in §L.C.1 to define the improvement’s measurement.

Q3. In 3.1.4 §L.D.3, the rollback was triggered by a sustained fall in success, a rise in interventions, a force violation, or a latency overrun. Which of the four are the regression check’s and which are the safety check’s?

Answer. The sustained fall and the latency overrun are the regression’s — the improvement’s performance side effects. The force violation and the intervention rise are the safety’s — the improvement’s safety side effects. Acceptance requires both the metric’s rise and the regression’s absence. Used in §L.C.2 to define the acceptance.

Q4. In 3.4.4 §L.C.3, the adaptation loop re-fits the mapping when the monitor’s MMD crosses the threshold. What is the improvement’s analogue of the re-fit trigger, and why is that the proposal’s dependency?

Answer. The analogue is the next improvement’s trigger — the residual’s crossing — and it is the dependency because one improvement’s residual is the next proposal’s input. The re-fit’s timing is what makes the ordering matter. Used in §L.B.3 to define the dependencies.

Part A — The proposal’s anatomy

§L.A.1 — The wish and the proposal

A wish is a statement of the desired state: “the shift survival should be at least 0.90”. A proposal is a statement of the change, the predicted effect, the cost, the measurement, and the kill criterion. The difference is the falsifiability of the content.

The running case’s wish and proposal for the top residual.

The wish. “The shift survival should be at least 0.90, and the composed recovery rate is the binding constraint.”

The proposal. “Raise the composed recovery rate from 0.809 to 0.95 by re-identifying the recovery policy’s contact model at every reset. Predicted effect: the shift survival under the 4-reset schedule rises from $8.56 \times 10^{-22}$ to $3.72 \times 10^{-6}$ — the shortfall of 21.07 decades falls to 5.43, so the change removes 74.2% of the decades between the composed rate and the 1.0 ceiling. Probability of success 0.55: the mechanism is the contact-model drift the 3.4.4 §L.C.3 monitor tracks, and the rate’s rise depends on the re-identification holding across the interval. Cost: 18 hours — 14 of cell time, 4 of analysis. Measurement: the 4.3.1 §L.C.1 paired design on the version-1 suite, 4 seeds, 800 episodes per segment, R’s CI at the 95% level. Kill criterion: if the CI’s lower bound on R is below 0.90 — the first rate at which the survival gains a decade — the change is abandoned and the 4.4.3 unknown-unknowns entry is the fallback.”

The five fields are the change, the expected effect, the cost, the measurement, and the kill criterion. The wish has none of the four quantitative ones. The wish restates the residual; the proposal addresses it.

The proposal’s falsifiability. The expected effect is a number with a mechanism and a probability, and the kill criterion is a threshold on a measured quantity. A proposal whose expected effect is “better” and whose kill criterion is “if it doesn’t work” is a wish with a plan’s formatting. Note what the last paragraph does: it states a 74.2% reduction of the shortfall and, in the same breath, admits the remainder is five decades. That is not weakness. A proposal that hides the remainder is the one the committee dismantles.

Self-explanation 1. Why is the expected effect’s number the core of the proposal’s falsifiability?

Answer. Because the number is what the measurement can contradict, and the contradiction is the proposal’s evidence. A proposal whose expected effect is qualitative cannot be contradicted, and an uncontradictable proposal is a wish regardless of how much detail surrounds it.

§L.A.2 — The five fields

The five fields, with their forms.

The change field. The specific modification — the parameter, the module, the schedule — with the invariants the change does not touch. Form: “change X from A to B; Y and Z are unchanged.”

The expected-effect field. The predicted metric change, with the mechanism — the pathway — and the probability that the effect materializes. Form: “the metric M rises by δ, with probability p, because the mechanism is the pathway P.”

The cost field. The time, the compute, and the risk of the change breaking something. Form: “T hours of cell time, C hours of analysis, and risk R of a regression.”

The measurement field. The experiment’s protocol — the paired design, the seeds, the suite version, the CI. Form: “the paired design on suite version V, with seeds S and the CI at level L.”

The kill criterion field. The threshold at which the improvement is abandoned, and the fallback — the claim’s re-scope, the rollback, the next proposal. Form: “if the metric M is below T at the measurement, the improvement is abandoned and fallback F is the plan.”

The five fields’ completeness. A change without the invariants is unbounded; an effect without the mechanism is a guess; a cost without the risk is an underestimate; a measurement without the CI is a number; a kill without the fallback is a dead end.

§L.A.3 — The proposal’s relationship to the defense

The proposal is 4.4.1 §L.B.3’s vulnerability map’s response, and its relationship to the defense is the forward-looking section.

The defense’s limits section names the residuals; the next section is the proposals. The committee’s evaluation of the defense is mostly the proposals’ quality: a candidate who sees the residuals and has a concrete proposal is the candidate the committee accepts.

The proposal’s role. The proposal is not the claim’s strengthening — the claim is the evidence’s — it is the residual’s address, and the address is the mastery signal. The committee’s question “what would you do next?” is the proposal’s invitation, and the five fields are the answer.

Concept check. A candidate’s defense has a “next” section reading “we would improve perception and reasoning”. Is that a proposal?

Answer. No. It is a wish — two subsystems named, with no change, effect, cost, measurement, or kill. The fix is the five fields for one of the two, with numbers. [displaces M1]

Worked-example ladder.

WE-L.A.a (fully worked). The H4 register row has weight 15: I5 detects an unintended contact at 0.68 s against a 0.20 s hazard, a margin of minus 0.48 s. Write the five fields for its proposal.

Solution. Change: replace I5’s force-derivative detector with the 4.2.4 §L.D.2 quarter-cycle estimator, cutting detection from 0.68 s to 0.28 s; the 4.2.2 §L.B.1 four authorities and the 4.2.4 §L.A.2 five invariants are unchanged. Expected effect: the margin moves from −0.48 s to +0.28 s, so the detection deficit closes by 0.48 of the 0.48 s available range — a delta of 0.83 — with p = 0.75, because the estimator’s lead depends on the contact’s stiffness being in the band the 4.2.4 §L.B.4 table covers. Cost: 6 hours — 4 of cell time for the force-sweep calibration, 2 of analysis; the regression risk is that the derivative noise trips the detector on the gripper’s own servo ripple. Measurement: the 4.3.1 §L.C.1 paired design on the version-1 suite, 4 seeds, 400 episodes per contact class, the detection-latency CI at 95%. Kill: if the CI’s lower bound on the margin is below 0.20 s, the estimator is abandoned and the 4.2.4 §L.D.2 compliance-limit fallback is the plan.

WE-L.A.b (completion). Write the five fields for the recovery-effectiveness decay residual, weight 12.

Completion. Change: the evaluation schedule — the recovery rate re-measured every segment of the 4-reset shift, at 1.60-hour intervals, instead of once per shift; the 4.1.3 §L.E.1 strata and the 4.2.4 §L.E.3 register are unchanged. Effect: the decay rate is identified at ±5.9 points, with p = 0.80, because the per-segment CI resolves a drop of the size the 4.3.3 §L.C.3 compounding predicts. Cost: 22.2 hours of cell time and 2 hours of analysis. Measurement: the 4.3.1 §L.C.1 paired design, 800 episodes per segment, the per-segment R CI at 95%. Kill: if the per-segment CI’s half-width exceeds 0.10, the audit is redesigned — the half-width is the instrument’s resolution, and an instrument that cannot see a 5.9-point drop cannot certify the decay.

WE-L.A.c (faded). Write the five fields for the residual 4.4.1 ranks seventh, the extrapolation-contract violation (weight 8).

Faded step. Change: the evaluation from a single segment to the 4-segment schedule at the 2-hour maintenance intervals. Invariants: the maintenance schedule and the recovery’s five kinds unchanged. Effect: the contract’s clauses are tested, with p = 0.80. Cost: 22.2 hours of cell time. Measurement: the per-segment CI at 800 episodes per segment. Kill: if the per-segment CI is wider than 0.10, the design is revised.

Completion. The proposal is the multi-segment evaluation, and its effect is the contract’s clauses being tested — not the contract’s validity, which is 4.4.3’s unknown-unknowns content. Note the asymmetry with WE-L.A.b: the same evaluation serves two different residuals, and only one of them is worth a proposal of its own. That is what the weights are for.

WE-L.A.d (contrastive near-miss). A candidate’s proposal is “increase the training data to 10,000 episodes”. The committee asks “which metric, by how much, at what cost, and when would you stop?”. The candidate answers with the first field only. Why it fails: the proposal has the change but not the effect, the cost, the measurement, or the kill, and the committee’s questions are exactly the missing fields. The fix: the five fields, per §L.A.2.

Comparative table: the wish and the proposal.

Design principle P61.1. A proposal is five fields; the effect’s number and the kill’s threshold are the proposal’s falsifiability, and a wish with a plan’s formatting is still a wish.

Consolidation checkpoint 1

(a) Three capabilities. You can write the five fields for a residual. You can distinguish a wish from a proposal by the fields’ presence. You can state the proposal’s relationship to the defense’s next section.

(b) Transfer probe. A proposal’s expected effect is “the composed rate rises from 0.809 to 0.90, with p = 0.6”. Is the effect field complete? Answer: the number, the probability, and the rate’s direction are present; the mechanism is missing, and the mechanism is the field’s pathway. The fix is the mechanism statement — “because the contact model is re-identified at every reset, and the drift the 3.4.4 §L.C.3 monitor tracks is what pulls the rate down inside the interval”.

(c) Remediation rule. If you cannot write the five fields for a residual of your choice, re-read §L.A.2 before Part B.

Part B — The value-cost frontier

§L.B.1 — The expected value

The expected value of an improvement is the residual’s weight reduction times the probability that the improvement succeeds.

where $w_i$ is the residual’s weight, $\Delta_i$ the fraction of the weight the improvement removes, and $p_i$ the probability that the improvement succeeds.

The delta is the field that is most often asserted and least often computed, so this lesson computes it. For the recovery residual the weight is not a gap in the rate — the gap is 0.19049, and a rate is not a vulnerability. The weight is the 21.07 decades of shift survival between 1.0 and the composed rate, and an improvement that lifts the rate to $R^\star$ removes the fraction of those decades it takes back:

with $S_4(R)$ the 4-reset survival at escalation rate $q = p(1-R)$, $p = 0.18$. This is the lab’s delta_recovery, and it has the two properties a fraction must have: it is 0 when $R^\star$ equals the composed rate and 1 as $R^\star$ approaches the floor. At $R^\star = 0.95$ it is 0.7423.

The running case’s values.

The recovery redesign. $w = 25$, $\Delta = 0.7423$, $p = 0.55$: $V = 10.21$.

The I5 detection. $w = 15$, $\Delta = 0.8333$, $p = 0.75$: $V = 9.38$. Here the delta is the margin recovered against the available range: the deficit is 0.48 s and the estimator closes 0.40 s of it, so $\Delta = 0.40/0.48 = 0.8333$.

The per-segment audit. $w = 12$, $\Delta = 0.50$, $p = 0.80$: $V = 4.80$. Half the decay weight is removed, not all of it: the audit identifies the decay, and identification is half of what a residual’s weight asks for. The other half is the treatment, which is the redesign.

The expected value’s interpretation. $V$ is the residual-risk reduction, in the vulnerability map’s weight units. It is not metric points; it is defense strength.

§L.B.2 — The value-cost ratio

The value-cost ratio is $V_i / C_i$, with $C_i$ the cost in the same units — here the hours of cell time plus engineering time.

The running case’s ratios.

The I5 detection. $C = 6$ hours; $V/C = 1.5625$ per hour.

The recovery redesign. $C = 18$ hours; $V/C = 0.5670$ per hour.

The per-segment audit. $C = 22.2$ hours; $V/C = 0.2162$ per hour.

The ratio’s ranking: the I5 detection first, the recovery redesign second, the per-segment audit third. The ranking follows the ratio, not the effect size — and note that the residual with the largest weight is not the one with the largest ratio. A 25-weight residual at 18 hours loses to a 15-weight residual at 6 hours, because 25/18 is less than 15/6. This is the single most useful thing the frontier tells a candidate, and it is the thing a candidate defending the biggest number gets wrong.

The ratio’s frontier. Plot the proposals on the value-cost plane; the frontier is the set of proposals no other dominates — higher value at lower cost. A dominated proposal is dropped from the portfolio. Here nothing is dominated: the I5 detection has the highest ratio, the recovery redesign the highest value, and each is best on one axis. The frontier has two points, and both survive Part B’s ranking.

§L.B.3 — The dependencies

The dependencies between improvements are the ordering constraints, and the ordering is not the value-cost ratio.

The running case’s dependencies.

The per-segment audit enables the recovery redesign’s effect to be read. The redesign claims its effect as the removal of 74.2% of the survival decades, and that claim is about the rate holding across the interval. If the rate decays within the interval, the redesign’s measured effect is the redesign’s gain minus the decay it did not treat — and the audit is what separates them. Run the redesign without the audit and the 74.2% figure is uninterpretable. The audit is the prerequisite.

The I5 detection has no prerequisite. Its residual is a per-episode margin, and its measurement is a contact-class sweep; nothing in the other two proposals changes what its CI measures.

The recovery redesign feeds 4.4.3’s unknown-unknowns entry. The remainder — 5.43 decades, a survival of $3.72 \times 10^{-6}$ against a 0.90 floor — is the unknown-unknowns artefact’s forward-looking content, because it is the residual that no available improvement closes.

The dependency’s consequence. The portfolio’s plan is the topological order of the dependency graph, tie-broken by the value-cost ratio. The running case’s plan: the I5 detection (first, no dependencies, highest ratio), the per-segment audit (second, no dependencies but a lower ratio), the recovery redesign (third, depends on the audit).

The dependency’s failure. A portfolio that runs the redesign before the audit has the decay inside its own measured effect, and the redesign’s evidence overstates the gain by however much the rate falls. The violation is not a crash; it is a number that looks better than the change deserves.

§L.B.4 — The portfolio’s selection

The portfolio is the set of proposals the defense commits, and the selection is the budget constraint.

The running case’s budget is the 40 hours of 4.4.2 preparation, with a 10-hour reserve for the 4.3.2 §L.D.3 diagnosis if proposals fail. The usable budget is therefore 30 hours. Walking the topological order: the I5 detection (6.0 hours) fits, leaving 24; the per-segment audit (22.2 hours) fits with 1.8 to spare; the recovery redesign (18.0 hours) does not.

The portfolio’s content. The committed proposals are the I5 detection and the per-segment audit, at 28.2 hours; the deferred is the recovery redesign, and the deferral is 4.4.3’s forward-looking content. The budget’s parts: 28.2 committed + 10 reserve + 1.8 slack = 40.

The deferral is the lesson’s sharpest moment. The residual the map ranks first is the one the portfolio defers — not because it is unimportant, but because at 18 hours it does not fit beside the audit that makes it interpretable, and the audit at 22.2 hours is the one that has to be paid for first. A candidate who cannot say this out loud has a portfolio, not a plan.

Self-explanation 2. Why is the value-cost ranking not the portfolio’s order?

Answer. Because the dependencies order the proposals and the ratio orders the independent ones. Here the two orders genuinely differ: the ratio ranks the audit third and the redesign second, and the dependency forces the audit before the redesign. The ratio breaks ties among proposals that are ready at the same time; it does not decide the sequence, and a candidate who lets it decide runs the redesign without the instrument that reads it.

Concept check. The per-segment audit’s ratio is 0.2162 and the recovery redesign’s is 0.5670, and the redesign depends on the audit. Can the redesign run first?

Answer. No. The redesign’s effect is the fraction of survival decades removed by the rate holding across the interval; without the audit the decay sits inside that measurement, and the effect is uninterpretable. The ratio ranking applies after the dependency, not before. [displaces M3]

Worked-example ladder.

WE-L.B.a (fully worked). The three proposals have $V$ values 10.21, 9.38, and 4.80, and $C$ values 18.0, 6.0, and 22.2. Compute the ratios and the ranking.

Solution. The ratios are 0.5670, 1.5625, and 0.2162. The ranking is the I5 detection first, the recovery redesign second, the per-segment audit third — and the portfolio’s order, after the dependency, is the I5 detection, the audit, the redesign.

WE-L.B.b (completion). The recovery redesign’s $p$ falls to 0.40 because the contact-model re-identification is less reliable than the 4.3.3 §L.C.3 monitor suggests. Recompute the $V$ and the ratio, and say whether the ranking changes.

Completion. $V = 25 \times 0.7423 \times 0.40 = 7.42$; the ratio is 0.4124. The ranking is unchanged — the redesign stays second, because 0.4124 still exceeds the audit’s 0.2162 — and so is the portfolio, whose selection the dependency already fixed. This is worth noticing: the estimate the change’s probability deserves is uncertain, and on this range the uncertainty does not move any decision. The proposal is robust to its own weakest field.

WE-L.B.c (faded). Compute the portfolio with a 40-hour budget and a 10-hour reserve.

Faded step. The usable budget is 30 hours. The I5 detection’s 6.0 hours fit; the audit’s 22.2 hours fit with 1.8 to spare; the redesign’s 18.0 hours do not. The committed 28.2 hours plus the 10-hour reserve leaves 1.8 hours of slack.

Completion. The portfolio commits two and defers one, and the deferral — not the ranking — is what the committee will ask about.

WE-L.B.d (contrastive near-miss). A team ranks by effect size and runs the recovery redesign first. The redesign’s measured effect is larger than the proposal predicted, because the segment over which the rate was measured also saw the rate decay, and the two gains are added together in the same number. Why it fails: the effect size is not the value, and the dependency is violated in a way that flatters the result rather than exposing it. The fix: the audit first, per §L.B.3.

Comparative table: the three proposals.

Design principle P61.2. Rank by the value-cost ratio and order by the dependencies; the ratio is the priority, the dependency is the sequence, and the portfolio is the budget’s frontier. The largest weight does not give the largest ratio.

Consolidation checkpoint 2

(a) Three capabilities. You can compute the expected value and the value-cost ratio. You can identify the dependencies and order the portfolio. You can select a portfolio within a budget with a reserve.

(b) Transfer probe. A proposal has $w = 10$, $\Delta = 0.4$, $p = 0.9$, and costs 5 hours. Compute the value and the ratio. Answer: $V = 10 \times 0.4 \times 0.9 = 3.6$; the ratio is 0.72 per hour. The proposal’s ranking is by the ratio, not the weight.

(c) Remediation rule. If you cannot state why the dependency orders the proposals, re-read §L.B.3 before Part C.

Part C — The improvement’s protocol

§L.C.1 — The change and the invariants

The protocol’s first part is the change and the invariants — what moves and what stays.

The change. The specific parameter or module, with the old value, the new value, and the mechanism by which the new value produces the effect. The change is minimal: an improvement that changes three things is three improvements in one proposal, and the attribution of the effect to the change is lost.

The invariants. The list of properties the change does not touch, with the evidence that the change preserves each. The invariants are the 4.2.1 §L.A.1 contract’s five fields, the 4.2.2 §L.B.1 four authorities, the 4.2.4 §L.A.2 five invariants, and the 3.3.4 §L.B four-layer stack. The change is safe when the invariants are verified, not when the change promises to preserve them.

The regression check. The invariants are tested after the change, on the same suite, and the regression is an invariant’s violation. The 3.1.4 §L.D.3 rollback triggers are the regression checks, and acceptance is conditional on the checks passing.

The change and the invariants trade off. A change that preserves the invariants trivially — the no-op — has no effect; a change that maximizes the effect and breaks an invariant is a net loss. The protocol’s discipline is invariants first, effect second.

§L.C.2 — The acceptance’s three conditions

The acceptance has three conditions, and the improvement is accepted only when all three hold.

The metric’s rise. The improvement’s metric — the detection margin, the composed rate, the per-segment CI’s half-width — rises above the pre-registered threshold, at the pre-registered CI’s lower bound.

The regression’s absence. The invariants are verified and no 3.1.4 §L.D.3 rollback trigger fires. The regression’s absence is necessary, not sufficient, and a metric’s rise with a regression is a net decision.

The cost within the budget. The improvement’s actual cost is within the proposal’s cost field plus the reserve’s share. A cost overrun is the portfolio’s re-plan, not the improvement’s abandonment, unless the overrun exceeds the reserve.

The three conditions’ conjunction. The improvement is accepted when all three hold; the committee’s evaluation is the conjunction, and the improvement registry is the record.

§L.C.3 — The kill criterion

The kill criterion is the threshold at which the improvement is abandoned, and its discipline is the pre-registration.

The criterion’s form. “If the metric M is below T at the measurement, the improvement is abandoned, and fallback F is the plan.” T is the effect’s minimum, and F is the claim’s re-scope, the rollback, or the next proposal.

The criterion’s purpose. The criterion is the falsifiability — it is what makes the proposal’s evidence a test rather than a hope. Its presence is the proposal’s strength: the committee rewards the candidate who has pre-committed to abandonment, because that commitment is the honesty the committee needs.

The criterion’s calibration. T is the effect size that justifies the improvement’s cost, and it is the 4.3.1 §L.C.1 power arithmetic’s inverse — the effect detectable at the measurement’s size. A T too tight kills improvements that work; a T too loose accepts improvements that do not.

The criterion’s fallback. F is the plan for abandonment, and it is the proposal’s completeness — an improvement without a fallback is a dead end. The running case’s fallbacks: the I5 detection’s is the 4.2.4 §L.D.2 compliance-limit fallback; the audit’s is the redesign of the evaluation at a larger per-segment size; the redesign’s is the 4.4.3 unknown-unknowns entry.

Self-explanation 3. Why is the kill criterion’s pre-commitment the proposal’s strength rather than its weakness?

Answer. Because the pre-commitment is the falsifiability — the committee knows the improvement will be abandoned if the evidence contradicts it — and the possibility of contradiction is what makes the evidence a test. A proposal without a kill is unfalsifiable, and the unfalsifiable proposal is the one the committee rejects.

Concept check. The I5 detection’s kill criterion is “if the CI’s lower bound on the detection margin is below 0.20 s, the estimator is abandoned.” Is the criterion calibrated?

Answer. The margin available is 0.48 s (0.68 s of current detection against a 0.20 s hazard), and the proposal claims 0.40 s of it. A lower bound of 0.20 s is a claim that at least 42% of the deficit closes — above the MDE at 400 episodes per contact class, which the 4.3.1 §L.C.1 arithmetic puts at 5.7 points of a binary rate and which here corresponds to roughly 0.11 s of margin. The threshold sits above the noise floor and below the claim: calibrated. [displaces M5]

Worked-example ladder.

WE-L.C.a (fully worked). The I5 detection change replaces the force-derivative detector with the quarter-cycle estimator, and the metric is the detection margin. Write the three acceptance conditions.

Solution. The metric’s rise: the margin’s CI lower bound is at least 0.30 s, above the 0.20 s hazard. The regression’s absence: the 4.2.2 §L.B.1 four authorities and the 2.3.3 §L.B.2 force limit are verified, and the detector does not trip on the gripper’s own servo ripple. The cost: within 6.0 hours plus the reserve’s share.

WE-L.C.b (completion). The margin’s CI lower bound is 0.35 s and the pre-registered threshold is 0.40 s. Is the improvement accepted?

Completion. No. 0.35 is below 0.40, so the metric condition fails; the kill criterion is not the acceptance criterion — the kill is the abandonment threshold and the acceptance is the pre-registered threshold — and here they are the same number, 0.40 s, which is the design’s weakness rather than its strength. The improvement is not accepted, and the fallback is the redesign at a larger contact-class size.

WE-L.C.c (faded). Write the invariants list for the per-segment audit.

Faded step. The 4.2.1 §L.A.1 contract’s five fields, the 4.2.4 §L.A.2 five invariants, and the 3.3.4 §L.B four-layer stack. The audit’s change is the evaluation schedule, and the invariants are the system’s, unchanged.

Completion. The invariants are the system’s and the audit is the evaluation’s; the regression check is the system’s tests, and the schedule change does not touch them. The audit is the one proposal whose regression risk is genuinely near zero, which is most of why its ratio survives a 22.2-hour cost.

WE-L.C.d (contrastive near-miss). A proposal’s change is “improve perception and reasoning”. The protocol’s invariants are not listed and no regression check runs. The metric rises, and the 2.2.4 §L.B Q-shield’s rejection rate doubles. Why it fails: the change is not minimal — two subsystems — and the invariant is broken. The fix: the minimal change, the invariants list, and the regression check, per §L.C.1.

Comparative table: the acceptance’s three conditions.

Design principle P61.3. Specify the change and the invariants, accept on the three conditions, and pre-commit to the kill; invariants first and effect second, and the kill is the proposal’s falsifiability.

Consolidation checkpoint 3

(a) Three capabilities. You can write the change and the invariants for a proposal. You can state the acceptance’s three conditions. You can write the kill criterion with its fallback.

(b) Transfer probe. The margin’s rise is 0.45 s at the CI’s lower bound, the threshold is 0.40 s, and the 2.3.3 force limit is violated in the regression check. Is the improvement accepted? Answer: no. The regression’s absence is a necessary condition, and the force violation is the invariant’s breach. The improvement is a net loss, and the rollback is the 3.1.4 §L.D.3 one.

(c) Remediation rule. If you cannot state why the change must be minimal, re-read §L.C.1 before Part D.

Part D — Executing the improvement

§L.D.1 — The improvement registry

The improvement registry extends the 4.3.1 §L.A.2 six fields with the proposal’s five and the dependency, producing eight fields.

The fields.

1. The hypothesis. The predicted effect’s number, with its mechanism and its probability.
2. The change. The specific modification, with the invariants.
3. The cost. The time, the compute, and the risk.
4. The measurement. The paired design, the seeds, the suite version, the CI.
5. The kill. The threshold and the fallback.
6. The dependency. The prerequisite proposal and the ordering rationale.
7. The randomization. The episodes’ and seeds’ assignment procedure.
8. The analysis. The test, the $\alpha$, the power, and the multiple-comparison correction.

The registry’s purpose is the pre-registration, and the improvement registry is the 4.3.1 §L.A.2 registry’s specialization — the improvement’s claims are the effect’s, and the registry is the proposal’s contract.

§L.D.2 — The acceptance and the rollback

The acceptance’s execution has four steps, and the rollback is the fourth.

Step 1: the change’s implementation. The change is implemented on a branch, and the branch is the candidate. The invariants are tested on the branch first, before the measurement.

Step 2: the measurement’s execution. The paired design on the suite version, with the seeds and the CI. The measurement is the only field the improvement is judged by, and a post hoc metric is not the acceptance.

Step 3: the three conditions’ evaluation. The metric’s rise, the regression’s absence, and the cost are evaluated against the registry. The acceptance is the conjunction.

Step 4: the commit or the rollback. An accepted improvement is committed to main, and a rejected one is rolled back — the branch is discarded and the kill’s fallback is the plan. The rollback is the 3.1.4 §L.D.3 discipline, unchanged.

The rollback criteria — the 3.1.4 §L.D.3 four, restated — a sustained fall in success, a rise in interventions, a force violation, and a latency overrun. The improvement’s rollback uses the same triggers, and its kill is the pre-registered one: the kill is the measurement’s threshold and the rollback is the deployment monitor’s.

§L.D.3 — The four failure modes

The improvement’s four failure modes are the ways the improvement fails, and each has a signature.

The no-move. The metric does not rise — the effect is absent — and the improvement is rejected by the kill. The signature is the measurement’s CI centered on zero. The cause is a wrong mechanism or an effect below the MDE. The fix is the next proposal or the claim’s re-scope.

The net loss. The metric rises and an invariant breaks — the regression is present. The signature is the metric’s rise together with the regression check’s failure. The cause is a non-minimal change or an incomplete invariants list. The fix is a narrower change or the invariant’s addition.

The overrun. The metric rises, the regression is absent, and the cost exceeds the registry’s plus the reserve’s. The signature is the actual against the planned cost. The cause is a cost underestimate or unexpected work. The fix is the portfolio’s re-plan and the next proposal’s budget.

The dead end. The metric is inconclusive — the CI is too wide — and the kill is not triggered. The signature is a CI whose half-width exceeds the effect it is measuring. The cause is a measurement whose power sits below the effect. The fix is a larger measurement or a paired design.

The four modes’ diagnosis. The no-move is the effect’s absence; the net loss is an invariant’s breach; the overrun is the cost’s; the dead end is the measurement’s power. Each is a registry field — the effect, the invariants, the cost, the measurement — and the diagnosis is the field’s review.

Self-explanation 1. Why is the dead end the most expensive failure?

Answer. Because it is inconclusive: the measurement’s cost is spent and the evidence is absent, so the improvement is neither accepted nor clearly rejected. The re-measurement is the full cost again, and the portfolio is delayed by it. The running case has a live instance of exactly this, and §L.E.2 computes it.

Concept check. The improvement’s metric rises 5.5 points, the CI is $[0.5, 10.5]$, and the threshold is 5. Is the improvement accepted?

Answer. The CI’s lower bound is 0.5, below 5, so the improvement is inconclusive — the dead end. The fix is a larger measurement or a paired design, and acceptance waits. [displaces M4]

Worked-example ladder.

WE-L.D.a (fully worked). The I5 detection improvement is implemented and the measurement gives a margin of 0.62 s with CI $[0.22, 1.02]$. The threshold is 0.40 s. Is the improvement accepted?

Solution. The CI’s lower bound is 0.22, below 0.40, so the improvement is inconclusive — the dead end. The fix is a larger measurement — the 4.3.1 §L.C.1 power arithmetic at 800 episodes per contact class — or a paired design that cancels the contact-class variance, and acceptance waits.

WE-L.D.b (completion). The measurement is re-run at 800 episodes per class and the CI is $[0.36, 0.88]$. Is the improvement accepted?

Completion. The CI’s lower bound is 0.36, still below 0.40; the improvement remains inconclusive. The completion shows that the 0.40 s threshold sits above the effect’s lower reach at 800 episodes, and the threshold is the issue: the estimate is 0.62 s with the CI’s lower bound at 0.36. Either the threshold moves to 0.30 s, or the claim’s re-scope is the fallback. The improvement is not accepted on the registered criterion, and that is the discipline working.

WE-L.D.c (faded). Write the four failure modes’ signatures.

Faded step. No-move: the CI centered on zero. Net loss: the metric’s rise with a regression. Overrun: the cost above the budget. Dead end: the CI wider than the effect.

Completion. The four are the registry’s four fields — the effect, the invariants, the cost, the measurement — and the diagnosis is the field’s review.

WE-L.D.d (contrastive near-miss). A team’s improvement raises the metric and the 4.2.4 §L.B.1 I1 force invariant is verified, so the team commits. The 4.3.3 §L.E.2 shift survival falls to $1.7 \times 10^{-28}$. Why it fails: the regression check is the four invariants and I1 is one of them — the shift survival is the horizon invariant and the team never checked it. Hill-climbing on a metric, checking only the constraint you were thinking about, is the exact machine that produces this. The fix: the complete invariants list, per §L.C.1.

Comparative table: the four failure modes.

Design principle P61.4. Register the eight fields, accept on the conjunction, roll back on the triggers, and diagnose the four modes by the registry’s fields; the dead end is the most expensive, and its fix is the power’s.

Consolidation checkpoint 4

(a) Three capabilities. You can write the improvement registry’s eight fields. You can execute the acceptance’s four steps. You can diagnose the four failure modes from their signatures.

(b) Transfer probe. The improvement’s metric rises and the cost is 20% above the registry’s plus the reserve’s. Which mode, and what is the fix? Answer: the overrun. The fix is the portfolio’s re-plan and the next proposal’s budget, not the improvement’s abandonment — unless the overrun exceeds the reserve.

(c) Remediation rule. If you cannot state why the dead end is the most expensive, re-read §L.D.3 before Part E.

Part E — The running case

§L.E.1 — The three proposals

The running case’s top three proposals, from 4.4.1 §L.B.3’s vulnerability map.

Proposal 1: the I5 detection speed-up. Weight 15. The change: replace the force-derivative detector with the 4.2.4 §L.D.2 quarter-cycle estimator, taking detection from 0.68 s to 0.28 s, with the 4.2.2 §L.B.1 four authorities and the 4.2.4 §L.A.2 five invariants unchanged. The effect: the margin moves from −0.48 s to +0.28 s, closing 0.40 s of the 0.48 s deficit — a delta of 0.8333 — with p = 0.75, because the estimator’s lead depends on the contact stiffness sitting in the band the 4.2.4 §L.B.4 table covers. The cost: 6.0 hours. The kill: if the CI’s lower bound on the margin is below 0.20 s, the estimator is abandoned and the 4.2.4 §L.D.2 compliance-limit fallback is the plan.

Proposal 2: the recovery redesign. Weight 25. The change: re-identify the recovery policy’s contact model at every reset, raising the composed rate from 0.809 to 0.95; the 4.2.4 §L.C.3 composition and the 4.3.3 §L.E.1 reset schedule are unchanged. The effect: the shift survival rises from $8.56 \times 10^{-22}$ to $3.72 \times 10^{-6}$, removing 74.2% of the 21.07 decades between the composed rate and the ceiling, with p = 0.55, because the mechanism is the contact-model drift the 3.4.4 §L.C.3 monitor tracks. The cost: 18.0 hours. The kill: if the CI’s lower bound on R is below 0.90 — the first rate at which the survival gains a decade, from 21.07 down to 10.92 — the change is abandoned and the 4.4.3 unknown-unknowns entry is the fallback.

Proposal 3: the per-segment recovery audit. Weight 12. The change: measure the recovery rate every segment of the 4-reset shift, at 1.60-hour intervals, instead of once per shift; the 4.1.3 §L.E.1 strata and the 4.2.4 §L.E.3 register are unchanged. The effect: the decay slope is identified at ±3.85 points — against a predicted drop of 5.9 points over the interval, so the signal separates — with p = 0.80. The cost: 22.2 hours of cell time and 2.0 of analysis. The kill: if the per-segment CI’s half-width exceeds 0.10, the evaluation is redesigned.

The dependencies. The audit enables the redesign’s effect to be read, because the redesign’s 74.2% is a claim about the rate holding across the interval. The I5 detection is independent of both. The redesign’s remainder feeds 4.4.3’s unknown-unknowns entry.

§L.E.2 — The portfolio’s plan

The budget is 40 hours and the reserve is 10.

The committed. The I5 detection (6.0 hours) and the per-segment audit (22.2 hours) total 28.2 hours. The reserve’s 10 hours leaves 1.8 hours of slack.

The deferred. The recovery redesign’s 18.0 hours does not fit. It is deferred, and the deferral is 4.4.3’s forward-looking content.

The sequence. The I5 detection first (6.0 hours, no dependency, highest ratio), the per-segment audit second (22.2 hours), the redesign deferred.

The decay’s arithmetic, stated honestly. The audit at 800 episodes per segment resolves a per-segment R to ±0.0385, and the predicted drop over the interval is 5.9 points. The signal separates by a factor of 1.5 against its half-width. At 400 episodes the half-width would be 0.0545 and the drop would not separate; the plan’s 800 is the smallest size that does. The 4.3.1 §L.C.1 arithmetic says the drop needs n ≥ 342 per segment, so 800 carries a margin rather than a bare pass. That is what a measurement field is for.

The redesign’s arithmetic, stated as honestly. The redesign removes 74.2% of the survival shortfall and leaves 5.43 decades: $3.72 \times 10^{-6}$ against the 0.90 floor. No available improvement closes that. The 4.4.1 §L.A.1 central claim already reports the number and names the shortfall as the residual, and this lesson’s contribution is that the proposal’s own arithmetic confirms the naming rather than pretending to repair it. The honest primary action is to state the residual; the redesign is the best available reduction, not a fix. A candidate who presents a 5.43-decade residual as a solved problem is the candidate the committee catches.

§L.E.3 — The register’s forward-looking entries

The proposals’ register entries, with the kill and the fallback.

The register’s entries are the 3.3.4 §L.E.3 five fields, and the fallbacks are 4.4.3’s unknown-unknowns content.

Self-explanation 2. Why is the redesign’s remainder — 5.43 decades — the proposal’s central honesty rather than its failure?

Answer. Because the remainder is what the arithmetic gives, and a proposal that reported a solved horizon would be reporting a number no mechanism supports. The floor needs $R \ge 0.99949$ under the constant-hazard form, and $R \ge 0.99958$ under the 4.3.3 drift law; the best available redesign reaches 0.95. The gap is a property of the system, not of the proposal, and the proposal’s job is to move the number and say by how much — which the 21.07 → 5.43 decade comparison does exactly.

Worked-example ladder.

WE-L.E.a (fully worked). The redesign’s target is $R = 0.95$, the composed rate is 0.809, the floor needs 0.99958 under the drift law. Compute the proposal’s status and the honest statement.

Solution. The survival at 0.95 is $3.72 \times 10^{-6}$, 5.43 decades below 1.0; at the composed rate it is $8.56 \times 10^{-22}$, 21.07 decades. The redesign removes $(21.07 - 5.43)/21.07 = 0.7423$ of the shortfall and leaves 5.43 decades, or a survival of $3.72 \times 10^{-6}$ against a 0.90 floor. The honest statement: the redesign is the largest single reduction available, and it does not reach the floor; the residual is named, not repaired.

WE-L.E.b (completion). The redesign’s re-identification holds better than expected and the composed rate reaches 0.99. Recompute the decades and say whether the floor is met.

Completion. The survival at 0.99 is $8.30 \times 10^{-2}$, 1.08 decades below 1.0, so the redesign removes $(21.07 - 1.08)/21.07 = 0.9487$ of the shortfall. The floor is still not met — 0.83 against 0.90 — and the remaining gap is 1.08 decades. The completion shows the shape of the problem: the survival is exponential in the rate, so the last decade costs as much rate as the first twenty did together, and each further decade costs more than the last.

WE-L.E.c (faded). Write the I5 detection’s kill fallback and the 4.4.3 unknown-unknowns entry that its residual feeds.

Faded step. The fallback is the 4.2.4 §L.D.2 compliance-limit fallback. The unknown-unknowns entry is the H4 residual’s remainder — the contact classes whose stiffness sits outside the 4.2.4 §L.B.4 band.

Completion. The fallback is the compliance limit, and the unknown-unknowns entry is the stiffness band’s edge, which is 4.4.3’s content.

WE-L.E.d (contrastive near-miss). A team’s portfolio commits the recovery redesign first, without the audit, and reports a survival gain larger than the proposal predicted. The gain is real, and it includes the decay the team never measured. Why it fails: the dependency is violated, and the violation inflates the evidence rather than exposing it. The fix: the audit first, per §L.B.3.

Comparative table: the portfolio’s plan.

Design principle P61.5. Commit the frontier within the budget and defer the rest; rank by the ratio, order by the dependency, and state the remainder the arithmetic leaves.

Consolidation checkpoint 5

(a) Three capabilities. You can state the three proposals’ fields. You can compute the portfolio within the budget. You can read the register’s forward-looking entries.

(b) Transfer probe. The redesign reaches $R = 0.95$ and the floor needs 0.99958. What is the proposal’s status and what is the honest statement? Answer: the redesign removes 74.2% of the survival shortfall and leaves 5.43 decades; it is not a fix, and the honest statement names the remainder as the residual rather than presenting the reduction as a solution.

(c) Remediation rule. If you cannot state why the redesign’s remainder is the honest content, re-read §L.E.2 before the lab.

Failure taxonomy

The table covers the three lab corruptions plus four failures not exercised in the lab: the inert change, the wrong metric, the scope creep, and the flattered effect.

The lab

Task. Write the improvement registry for the three proposals — the I5 detection, the per-segment audit, and the recovery redesign — compute the expected values, the value-cost ratios, and the dependencies, select the portfolio within the 40-hour budget, and run the acceptance’s three conditions on a synthetic measurement with a known ground truth.

Conditions. Python 3.10+, NumPy. The weights are 15, 12, and 25; the deltas are computed from the mechanics, not asserted: the recovery delta is the fraction of the survival decades removed, and the I5 delta is the fraction of the detection deficit closed. The measurement is a synthetic effect with a CI, and the acceptance is the three conditions’ conjunction. Seed all RNGs.

Controls. (i) The five-field portfolio; (ii) the portfolio without the dependencies; (iii) the portfolio without the kill criterion; (iv) the portfolio with the effect below the MDE.

Metrics.

Pre-registered hypotheses.

H1. The I5 detection has the highest V/C, at 1.5625, because its weight-to-cost ratio dominates — and the 25-weight redesign does not rank first, which is the frontier’s central lesson.

H2. The portfolio without the dependencies runs the redesign before the audit and reports a survival gain larger than the redesign’s own prediction — the dependency violation inflates the evidence.

H3. The portfolio without the kill criterion has unfalsifiable proposals, and the 4.4.1 §L.B.3 vulnerability map is unchanged — a proposal that cannot fail cannot reduce a residual.

H4. The redesign’s target of 0.95 leaves 5.43 decades and a survival of $3.72 \times 10^{-6}$ against the 0.90 floor, so the proposal is a reduction and not a fix — and the honest primary action is stating the residual.

Break-it sequence.

1. Remove the kill silently. The proposal is unfalsifiable and the committee’s rejection follows. Silent: the effect and the cost are present.
2. Run the redesign first silently. The measured effect absorbs the decay, and the reported gain exceeds the prediction. Silent: each proposal is individually valid, and the number looks better rather than worse.
3. Accept an inconclusive measurement silently. The CI includes zero, and the inconclusive reading is the improvement’s status. Silent: the point estimate is positive.

Acceptance criteria.

What good looks like. A good artefact has five properties. (1) The five fields are each a sentence with a number. (2) The delta is computed from the mechanism, not asserted. (3) The dependencies are a graph and the order is topological. (4) The kill and the fallback are paired. (5) The break-it results are a table, and the second one reports a number that moved the wrong way.

A runnable core of the ranking, the dependencies, the portfolio, and the acceptance, seeded and annotated:

# lab_4_4_2.py — proposals: value-cost, dependencies, portfolio, acceptance
import numpy as np
from math import log10, sqrt, ceil

# ---- the vulnerability map's top three (4.4.1 §L.B.3) -------------------
# weight = probability x consequence, 1..5 each, so w in 1..25
TOP3 = [
    ("recovery_redesign",  25, "4.3.3 H6: the composed rate is below the floor"),
    ("i5_detection",       15, "4.2.4 H4: I5 detects at 0.68 s against a 0.20 s hazard"),
    ("segment_recovery_audit", 12, "4.3.3 H2: recovery effectiveness decays in the interval"),
]

# ---- the horizon arithmetic the first residual is read from ------------
$P_FAIL$, $R_NOW$, $N_SHIFT$, $T_D$, FLOOR = 0.18, 0.809, 1152, 576.0, 0.90

def $shift_survival(n_resets$, q, $t_d=T_D$, $n=N_SHIFT$):
    """Drift hazard on $n_resets+1$ segments; the last segment absorbs the remainder."""
    assert q >= 0.0
    m = $n_resets$ + 1
    base = n // m
    ks = [base] * (m - 1) + [n - base * (m - 1)]
    assert sum(ks) == n, "the segments must tile the shift"
    return float(np.prod([np.prod(1.0 - q * (1.0 + np.arange(k) / $t_d$)) for k in ks]))

def required_recovery(fl=FLOOR, $n=N_SHIFT$, $p=P_FAIL$):
    """The R at which zero failures still meets the floor (constant hazard, 4.3.3 Eq A.3)."""
    return 1.0 - (1.0 - fl ** (1.0 / n)) / p

$Q_NOW$ = $P_FAIL$ * (1.0 - $R_NOW$)
SURV_NOW = shift_survival(4, $Q_NOW$)
$R_FLOOR$ = required_recovery()
DECADES_NOW = -log10(SURV_NOW)

# ---- the delta is computed, not asserted -------------------------------
def decades_to_floor(R):
    """Decades of shift survival between the floor and the rate."""
    return -log10(shift_survival(4, $P_FAIL$ * (1.0 - R)))

def $delta_recovery(R_target$):
    """The fraction of the survival shortfall the redesign removes.

    The residual is not a gap in R; it is the 21 decades of shift survival
    between 1.0 and the composed rate. A redesign to $R_target$ is worth the
    fraction of those decades it takes back.
    """
    return (DECADES_NOW - $decades_to_floor(R_target$)) / DECADES_NOW

DELTA_REC = delta_recovery(0.95)
DELTA_I5 = (0.68 - 0.28) / (0.68 - 0.20)      # margin recovered against the 0.20 s hazard

PROPOSALS = {
    "i5_detection":           {"w": 15, "delta": DELTA_I5,  "p": 0.75, "cost_h": 6.0},
    "segment_recovery_audit": {"w": 12, "delta": 0.50,      "p": 0.80, "cost_h": 22.2},
    "recovery_redesign":      {"w": 25, "delta": DELTA_REC, "p": 0.55, "cost_h": 18.0},
}
# the audit is the instrument the redesign's effect is read with
DEPS = {"i5_detection": [], "segment_recovery_audit": [],
        "recovery_redesign": ["segment_recovery_audit"]}
BUDGET_H, RESERVE_H = 40.0, 10.0

def value(p):
    assert 0.0 <= p["delta"] <= 1.0 and 0.0 <= p["p"] <= 1.0
    return p["w"] * p["delta"] * p["p"]

def ratio(p):
    return value(p) / p["cost_h"]

print("== canaries ==")
print("C1 the composed rate survives to %.3e, %.2f decades below 1.0"
      % (SURV_NOW, DECADES_NOW))
print("C2 the floor needs R >= %.5f, and the composed rate is %.3f = %.5f short"
      % ($R_FLOOR$, $R_NOW$, $R_FLOOR$ - $R_NOW$))
print("C3 delta is zero at the current rate      : %.3e" % $delta_recovery(R_NOW$))
print("C4 delta is one at the floor              : %.6f" % $delta_recovery(R_FLOOR$))
print("C5 one hundredth of R buys %.2f decades   : %.3f"
      % ($decades_to_floor(R_NOW$) - $decades_to_floor(R_NOW$ + 0.01),
         $decades_to_floor(R_NOW$) - $decades_to_floor(R_NOW$ + 0.01)))
print("C6 delta is monotone in the target        : %s"
      % all(delta_recovery(0.85 + 0.01 * i) < delta_recovery(0.86 + 0.01 * i)
            for i in range(10)))

print("\n== the survival the first residual is read from ==")
for R in ($R_NOW$, 0.90, 0.95, 0.99, $R_FLOOR$):
    print("  R=%.5f -> shift survival %.4e -> %.2f decades -> delta %.4f"
          % (R, shift_survival(4, $P_FAIL$ * (1.0 - R)), decades_to_floor(R),
             delta_recovery(R)))

print("\n== value-cost ranking ==")
ranking = sorted(PROPOSALS.items(), key=lambda kv: -ratio(kv[1]))
for name, p in ranking:
    print("  %-22s w=%2d d=%.4f p=%.2f  V=%6.3f  C=%5.1f  V/C=%6.4f"
          % (name, p["w"], p["delta"], p["p"], value(p), p["cost_h"], ratio(p)))

def topological_order(deps):
    """Kahn's algorithm; ready nodes are taken in descending value-cost ratio."""
    order, remaining = [], set(deps)
    while remaining:
        ready = [n for n in remaining if not (set(deps[n]) & remaining)]
        if not ready:
            raise ValueError("cycle in the dependency graph")
        ready.sort(key=lambda n: -ratio(PROPOSALS[n]))
        order.append(ready[0])
        remaining.remove(ready[0])
    return order

ORDER = topological_order(DEPS)
print$"\ntopological order:", ORDER$
print("the ratio's ranking and the dependency's order differ: %s"
      % (ORDER != [n for n, _ in ranking]))

def portfolio(budget_h, reserve_h):
    """Greedy over the topological order; a node is taken only if it fits with the reserve."""
    committed, spent = [], 0.0
    for name in ORDER:
        c = PROPOSALS[name]["cost_h"]
        if spent + c + reserve_h <= budget_h:
            committed.append(name)
            spent += c
    return committed, spent, budget_h - spent - reserve_h

committed, spent, slack = portfolio(BUDGET_H, RESERVE_H)
print("committed: %s  spent=%.1f  reserve=%.1f  slack=%.1f" % (committed, spent, RESERVE_H, slack))
deferred = [n for n in ORDER if n not in committed]
print("deferred : %s (%.1f h)" % (deferred, sum(PROPOSALS[n]["cost_h"] for n in deferred)))

def acceptance(ci_lower, threshold, regression_ok, cost_actual, cost_plan, reserve_share):
    """The three conditions are a conjunction; the metric condition reads the CI's lower bound."""
    return {"metric": ci_lower >= threshold,
            "regression": regression_ok,
            "cost": cost_actual <= cost_plan + reserve_share}

def verdict(a):
    return "accepted" if all(a.values()) else "rejected: " + ",".join(k for k, v in a.items() if not v)

print("\n== acceptance, I5 detection: 0.68 s -> 0.28 s, threshold 0.40 s ==")
a1 = acceptance(0.35, 0.40, True, 5.8, 6.0, 0.5)
print("  ci_lower 0.35 vs threshold 0.40 ->", a1, verdict(a1))
a2 = acceptance(0.42, 0.40, True, 5.8, 6.0, 0.5)
print("  ci_lower 0.42 vs threshold 0.40 ->", a2, verdict(a2))
a3 = acceptance(0.42, 0.40, False, 5.8, 6.0, 0.5)
print("  ci_lower 0.42, regression broken ->", a3, verdict(a3))

print("\n== the audit's power: is the decay signal detectable? ==")
DECAY_DROP = $R_NOW$ - 0.75
for $n_seg$ in (400, 800, 1600):
    sd = $sqrt(R_NOW$ * (1.0 - $R_NOW$) / $n_seg$)
    hw = 1.96 * sd * sqrt(2.0)
    print("  n=%4d per segment: R sd %.4f, half-width on the drop %.4f (%.2f points)"
          % ($n_seg$, sd, hw, 100 * hw))
print("  the drop R 0.809 -> 0.750 over the interval is %.3f (%.1f points)"
      % (DECAY_DROP, 100 * DECAY_DROP))
$n_needed$ = ceil(2.0 * (1.96 * $sqrt(R_NOW$ * (1 - $R_NOW$)) / DECAY_DROP) ** 2)
print("  the drop needs n >= %d per segment; the plan uses 800" % $n_needed$)

Lines 1–11 are the top three residuals from 4.4.1 §L.B.3, with their weights. Lines 13–42 the horizon arithmetic and the two computed deltas. Lines 44–58 the proposals, the dependencies, and the budget. Lines 60–82 the value, the ratio, and the ranking. Lines 84–98 the topological order. Lines 100–112 the portfolio. Lines 115–135 the acceptance’s conjunction. Lines 137–146 the audit’s power.

Measured output.

== canaries ==
C1 the composed rate survives to 8.565e-22, 21.07 decades below 1.0
C2 the floor needs R >= 0.99949, and the composed rate is 0.809 = 0.19049 short
C3 delta is zero at the current rate      : 0.000e+00
C4 delta is one at the floor              : 0.997395
C5 one hundredth of R buys 1.13 decades   : 1.126
C6 delta is monotone in the target        : True

== the survival the first residual is read from ==
  R=0.80900 -> shift survival 8.5645e-22 -> 21.07 decades -> delta 0.0000
  R=0.90000 -> shift survival 1.2072e-11 -> 10.92 decades -> delta 0.4817
  R=0.95000 -> shift survival 3.7235e-06 -> 5.43 decades -> delta 0.7423
  R=0.99000 -> shift survival 8.2973e-02 -> 1.08 decades -> delta 0.9487
  R=0.99949 -> shift survival 8.8131e-01 -> 0.05 decades -> delta 0.9974

== value-cost ranking ==
  i5_detection           w=15 d=0.8333 p=0.75  V= 9.375  C=  6.0  V/C=1.5625
  recovery_redesign      w=25 d=0.7423 p=0.55  V=10.207  C= 18.0  V/C=0.5670
  segment_recovery_audit w=12 d=0.5000 p=0.80  V= 4.800  C= 22.2  V/C=0.2162

topological order: ['i5_detection', 'segment_recovery_audit', 'recovery_redesign']
the ratio's ranking and the dependency's order differ: True
committed: ['i5_detection', 'segment_recovery_audit']  spent=28.2  reserve=10.0  slack=1.8
deferred : ['recovery_redesign'] (18.0 h)

== acceptance, I5 detection: 0.68 s -> 0.28 s, threshold 0.40 s ==
  ci_lower 0.35 vs threshold 0.40 -> {'metric': False, 'regression': True, 'cost': True} rejected: metric
  ci_lower 0.42 vs threshold 0.40 -> {'metric': True, 'regression': True, 'cost': True} accepted
  ci_lower 0.42, regression broken -> {'metric': True, 'regression': False, 'cost': True} rejected: regression

== the audit's power: is the decay signal detectable? ==
  n= 400 per segment: R sd 0.0197, half-width on the drop 0.0545 (5.45 points)
  n= 800 per segment: R sd 0.0139, half-width on the drop 0.0385 (3.85 points)
  n=1600 per segment: R sd 0.0098, half-width on the drop 0.0272 (2.72 points)
  the drop R 0.809 -> 0.750 over the interval is 0.059 (5.9 points)
  the drop needs n >= 342 per segment; the plan uses 800

Cheat sheet

Exercises with solutions

E 4.4.2.1. Write the five fields for an improvement that reduces the mapping’s drift.

S 4.4.2.1. Change: the 3.4.4 §L.C.3 monitor’s re-fit goes from monthly to weekly. Invariants: the 4.2.1 §L.B.3 checks and the 4.2.4 §L.B.1 I2 invariant. Effect: the MMD drift residual falls by 40%, with p = 0.75, because the re-fit interval shortens below the drift’s correlation time. Cost: 8 hours of cell time and 2 of analysis. Measurement: the 4.1.3 §L.C.3 MMD on the version-1 suite. Kill: if the residual falls by less than 20%, the improvement is abandoned and feature-level alignment is the fallback. The lesson: the five fields are the residual’s address, and the wish is the residual’s restatement.

E 4.4.2.2 (transfer). A proposal has $w = 12$, $\Delta = 0.5$, $p = 0.8$, and costs 6 hours. Compute the value and the ratio.

S 4.4.2.2. $V = 12 \times 0.5 \times 0.8 = 4.8$; the ratio is 0.80 per hour. The lesson: the value is the weight’s reduction, and the ratio is the priority.

E 4.4.2.3. State the dependencies of the three running-case proposals and the topological order.

S 4.4.2.3. The per-segment audit enables the recovery redesign’s effect to be read; the I5 detection is independent. The order is the I5 detection, the audit, the redesign. The lesson: the order is the dependency’s and the ratio is the tiebreaker, and here the two rankings genuinely differ.

E 4.4.2.4 (diagnostic). The metric’s rise is 3 points, the CI is $[0.5, 5.5]$, and the threshold is 5. Is the improvement accepted?

S 4.4.2.4. No. The CI’s lower bound is 0.5, below 5, so the improvement is inconclusive — the dead end. The fix is a larger measurement or a paired design. The lesson: acceptance is the CI’s lower bound, and the point estimate is not the decision.

E 4.4.2.5. Write the acceptance’s three conditions for the I5 detection proposal.

S 4.4.2.5. The metric’s rise: the margin’s CI lower bound is at least 0.30 s against a 0.20 s hazard. The regression’s absence: the 4.2.2 §L.B.1 four authorities and the 2.3.3 §L.B.2 force limit are verified, and the detector does not trip on the servo ripple. The cost: within 6.0 hours plus the reserve’s share. The lesson: the three are a conjunction, and the regression’s absence is necessary.

E 4.4.2.6 (transfer). The recovery redesign reaches $R = 0.95$ against a composed rate of 0.809. Compute the decades removed and state the proposal’s status.

S 4.4.2.6. The survival goes from $8.56 \times 10^{-22}$ (21.07 decades) to $3.72 \times 10^{-6}$ (5.43 decades), removing 74.2% of the shortfall and leaving 5.43 decades against a 0.90 floor. It is a reduction, not a fix. The lesson: the effect field’s number includes the remainder, and a proposal that omits the remainder is reporting a mechanism it does not have.

E 4.4.2.7 (diagnostic). The improvement’s metric rises and the 4.2.4 §L.B.1 I1 force invariant is verified. The shift survival falls to $1.7 \times 10^{-28}$. Diagnose and fix.

S 4.4.2.7. The net loss. The invariants list is incomplete — the shift survival is the 4.3.3 §L.B.1 long-horizon invariant and it was not checked. The fix is the complete invariants list and the narrower change. The lesson: the regression check must cover 4.3.3’s horizon invariants, not only 4.2.4’s within-episode ones.

E 4.4.2.8. Write the register’s entry for the deferred redesign with its fallback.

S 4.4.2.8. The proposal: the recovery redesign. The expected effect: the shift survival rises from $8.56 \times 10^{-22}$ to $3.72 \times 10^{-6}$, 74.2% of the shortfall removed and 5.43 decades remaining. The kill: the CI’s lower bound on $R$ at least 0.90. The fallback: the 4.4.3 unknown-unknowns entry. The lesson: the register’s forward-looking entries are 4.4.3’s content.

E 4.4.2.9 (transfer). The budget is 40 hours and the reserve is 10. The committed proposals total 28.2 hours. Is the redesign’s 18.0 hours within the budget?

S 4.4.2.9. No: 28.2 + 18.0 + 10 = 56.2, above 40. The redesign is deferred, and the deferral is 4.4.3’s forward-looking content. The lesson: the portfolio is the budget’s frontier, and the deferral is the plan — the top-ranked residual is the one deferred, and saying so is the plan.

Exit test

You are ready for Lesson 4.4.3 when you can, without notes:

1. State the difference between a wish and a proposal and write the five fields for a residual.
2. Compute the expected value from the weight, the delta, and the probability.
3. Compute a delta from the mechanism rather than asserting it, as Eq. (B.2) does for the recovery redesign.
4. Compute the value-cost ratio and rank the portfolio, and explain why the largest weight need not give the largest ratio.
5. Identify the dependencies and order the portfolio topologically.
6. Select a portfolio within a budget with a reserve and state the deferral.
7. Write the change and the invariants for a proposal.
8. State the acceptance’s three conditions.
9. Write the kill criterion with its fallback.
10. State the improvement registry’s eight fields.
11. Execute the acceptance’s four steps.
12. Name the four failure modes and their signatures.
13. State why the dead end is the most expensive.
14. State the running case’s three proposals and their statuses.
15. State why the redesign’s remainder — 5.43 decades — is the honest content of the proposal rather than its failure.

Cross-references. 4.4.1 supplies the vulnerability map, the residuals’ weights, and the “what would change my mind” statement restated in §L.A.1 and §L.E.3. 4.3.3 supplies the horizon’s failure law, the maintenance schedule, and the extrapolation contract restated in §L.B.2 and §L.E.2. 4.3.2 supplies the diagnosis’s stopping rule and the evidence chain restated in §L.B.3. 4.3.1 supplies the registry’s six fields, the power arithmetic, and the CI’s lower bound restated in §L.C.1 and §L.D.1. 4.2.4 supplies the register’s residuals and the recovery’s five kinds restated in §L.D.3. 3.4.4 supplies the adaptation loop and the mapping’s drift restated in §L.E.2. 3.1.4 supplies the decision conjunction and the rollback restated in §L.D.2. Lesson 4.4.3 (communicating the unknown unknowns) uses the improvement registry’s residuals as the unknown-unknowns artefact’s forward-looking section.

§L.10 — Provenance

Evidence tiers. A: reproducible from the shipped lab in this lesson or a cited lesson’s shipped lab. B: computed here from a cited lesson’s stated constants. C: cited to a lesson without recomputation.

Reproduction. Extract the block and run it under python3 -I; the canaries print first, and C3 and C4 are the delta’s two endpoints. The measured-output block above is the actual run, not a transcription.

Corrections to the earlier draft. Seven defects, numbered.

1. The proposal set was built on residuals that do not exist. The draft named “the success rate below the 0.90 target (weight 15)”, “the specification’s gap (weight 9)”, and “the horizon extrapolation (weight 9)”. None of the three is in 4.4.1 §L.B.3’s corrected map, which has one entry at 25 (the recovery rate below the floor), one at 15 (the H4 I5 detection margin), and a two-way tie at 12 resolved by a stated criterion to the recovery-effectiveness decay. Two of the three proposals addressed nothing the map ranks. Rebuilt against the map’s actual top three.

2. The delta was asserted, never computed. The draft gave $\Delta = 1.0$, 0.6, and 0.5 as round numbers with no derivation. The deltas here are computed: the recovery delta is the fraction of the 21.07 survival decades the target rate removes, via Eq. (B.2), and the I5 delta is the fraction of the 0.48 s detection deficit the estimator closes. Round numbers are what a delta looks like when nobody derived it.

3. The CI half-width claim was wrong by about a factor of two, in the wrong direction. The draft said the change from 1 seed to 4 seeds at 800 episodes narrows the interval “from 0.025 to 0.020”. At the running case’s success-rate scale the half-width is 0.0361 at 1 seed and 0.0256 at 4 seeds — a 29% narrowing, not 20% — and 0.020 requires roughly 3,582 episodes. The 0.020 figure was reported as reachable at 800 and is not.

4. The expected effect was stated two ways, tenfold apart. §L.A.1 said the lower bound rises 0.898 → 0.900, which is 0.2 points; §L.E.2 said the proposal’s effect is 2 points. The ratio is 10. The lesson no longer makes a success-rate proposal at all: the map’s top three are the recovery rate, the I5 margin, and the recovery decay, and the success-rate re-scope belongs to 4.4.1 §L.A.1’s claim statement rather than to this lesson’s portfolio.

5. The dead-end diagnosis rested on a comparison the arithmetic does not support. The draft compared a 2-point effect against a 4.1-point MDE and declared the dead end. The 2 points were invented; the MDE was real. The lesson now states a dead end that the arithmetic produces: the redesign removes 74.2% of the shortfall and leaves 5.43 decades, so the floor stays unmet and the residual is named rather than repaired.

6. The topological order and the ratio ranking were the same list, which made the dependency section vacuous. The draft’s ratio ranked the spec writer first and its dependency order also put the spec writer first, and its own Self-explanation 2 conceded the coincidence. Here the two genuinely differ — the ratio ranks the audit third and the redesign second, and the dependency forces the audit first — which is what makes §L.B.3 and §L.B.4 carry content.

7. The portfolio deferred nothing that mattered. The draft deferred a proposal whose residual the map ranks third, and committed the one at 15 and the one at a weight the map does not contain. Here the portfolio defers the residual the map ranks first, because the instrument that makes its effect readable costs more than it does, and saying that out loud is the plan.

Open item. The lesson cites 3.3.4 §L.B and §L.E.3 as the four-layer stack and the register’s five fields. 3.3.4 does not exist in the curriculum directory while being cited as a load-bearing prerequisite across many lessons. The citations are retained because removing them would silently drop a prerequisite rather than repair the gap, and the gap is flagged here rather than papered over.