Nyimbi Odero

Series 01 · a numbered run · kept in public

Lesson 4.3.3 — Long-Horizon Robustness

Chapter: Phase 4 — Synthesis and Mastery · Module 4.3 — Iteration, Evaluation, and Failure Analysis

Week 40

Prerequisite lessons: 4.3.2 (diagnosing sim-to-real failures), 4.3.1 (systematic experimentation and ablations), 4.2.4 (safety and recovery), 4.2.2 (policy layer), 3.2.3 (planning with world models), 1.1.4 (the full-stack pick-and-place pipeline), 3.3.4 (safety and constraints on LLM outputs).

Prior results used in this lesson. From 4.3.2: the stopping rule and the residual inside tolerance, restated in §L.A.3. From 4.3.1: the power arithmetic and the confidence-interval lower bound, restated in §L.B.2. From 4.2.4: the five recovery kinds, the composed recovery rate, and the register residuals, restated in §L.C.3. From 4.2.2: the total blind window and the attribution procedure, restated in §L.C.2. From 3.2.3: the usable horizon and the coverage flag, restated in §L.C.1. From 1.1.4: the seven-stage pipeline health checks and the state-machine recovery, restated in §L.A.2. From 3.3.4: the register residuals and the safe state, restated in §L.E.3. Each is restated at the point of use.

Hands forward. The horizon failure law, the long-horizon metrics, the drift maintenance policy, and the four degradation modes. Module 4.4 (the master’s defence) uses the horizon failure law and the maintenance policy as its forward-looking evidence.

The running case in one sentence. The arm, gripper, eye-in-hand camera, table, cube, bin, divider, and three distractors are unchanged; one pick-and-place takes 25 seconds, and the cell must run an 8-hour shift — 1,152 consecutive episodes — so this lesson asks how the success rate decays over that horizon, why it decays, and when maintenance must intervene before the decay crosses the deployment floor.


§L.0 — Lesson map, objectives, and misconceptions

PartContentLab stepWord budget
AThe compounding law and what the floor requiresCompute the horizon decay1,900
BLong-horizon metrics and window sizingCompute the window2,000
CDrift sources and the maintenance policyRun the drift injection1,900
DRecovery across horizons, degradation modes, schedule designCompute the recovery compounding1,800
ERunning case: the 8-hour shift planAssemble the maintenance plan1,600

Observable outcomes. By the end of this lesson you can, without notes: (1) derive the horizon failure law from the per-episode rate and state its three regimes; (2) compute the horizon at which a per-episode rate crosses a deployment floor, and say what recovery rate the floor actually demands; (3) distinguish independent from correlated failures and state what the correlation does to the tail; (4) name the five long-horizon metrics and their bounds; (5) compute the evaluation window from a confidence-interval resolution; (6) name the four drift sources, their detection channels, and their time constants; (7) state the maintenance policy’s three thresholds and their decisions; (8) state the recovery compounding law and the per-segment failure rate that follows from it; (9) name the four degradation modes and their signatures; (10) assemble the 8-hour shift plan with its maintenance schedule and register entries.

Misconception inventory.

Wrong modelWhy attractiveWhere displacedCorrect model
A high success rate means long-horizon robustness.The rate is the headline metric.§L.A.1The per-episode rate is the base of a power law. At 0.82 per episode over 1,152 episodes the independent survival is 0.821152=5.2×10−1000.82^{1152} = 5.2 \times 10^{-100}; the headline number says nothing about the shift until you raise it to the horizon.
Failures are independent across episodes.Each episode starts from a fresh reset.§L.A.2Failures cluster because the environment drifts. At the running case’s drift constant the linear-hazard survival is 1.8×10−361.8 \times 10^{-36} against the constant-hazard 3.1×10−183.1 \times 10^{-18} — a factor of 101810^{18} — and no reset schedule recovers the gap.
The evaluation window is the shift.The shift is the deployment horizon, so measure the shift.§L.B.2The window is sized by the confidence-interval resolution, not by the deployment target. At a five-point resolution on a 0.90 proportion the window is 138 shifts — 46 days of continuous operation — which is infeasible, so the shift is extrapolated from an episode-level suite under a stated contract.
Drift is slow degradation you can ignore.The system works at the start of the shift.§L.C.1Drift is the mechanism that produces correlated failures, and the maintenance interval is set by the smallest of the four drift constants — 4 hours in the running case, not the 8-hour camera constant the intuition reaches for.
Recovery is a per-episode property.The recovery rate is quoted per episode.§L.C.3Recovery compounds within a segment. The composed rate R=0.809R = 0.809 gives a per-episode escalation probability q=p(1−R)=0.0344q = p(1-R) = 0.0344, and a 48-episode segment survives with probability (1−q)48=0.187(1-q)^{48} = 0.187. The per-segment failure is 0.81, not “essentially certain” and not R48R^{48}.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.


§L.0.5 — Retrieval warm-up

Q1. In 4.3.2 §L.C.3 the stopping rule stopped when the residual fell inside tolerance. What does that residual represent for long-horizon robustness, and why is it the seed of drift?

Answer. The residual is the unattributed part of the sim-to-real gap: whatever the identified parameters failed to explain. It is the seed of drift because an unattributed residual is a state dependence on the environment, and drift is exactly the environment moving through that dependence. The residual inside tolerance is the current acceptance; the long-horizon question is whether the residual grows. Used in §L.A.3 to place drift inside the diagnosis residual.

Q2. In 4.2.4 §L.C.3 the composed recovery rate was 0.809 per episode. What does the long horizon ask of that number?

Answer. Whether 0.809 holds across the horizon, or whether the recovery’s own failures accumulate. The per-episode reading is that 80.9% of detected failures are recovered without escalation; the horizon reading is that the unrecovered fraction, 1−0.809=0.1911 - 0.809 = 0.191, multiplies the per-episode failure rate into an escalation rate, and that rate compounds over the shift. Used in §L.C.3 to define the recovery compounding.

Q3. In 4.2.2 §L.B.1 the layer’s total blind window was 1.2 s. What does that window contribute to the horizon failure law, and how is it different from the per-episode success rate?

Answer. The blind window bounds reactivity within an episode, so it contributes to the per-episode failure rate. The horizon law compounds across episodes. The two are nested: the per-episode rate is the consequence of the blind window plus every other cause, and the horizon law raises that consequence to a power. Used in §L.A.2 to separate within-episode from across-episode compounding.

Q4. In 3.2.3 §L.D.1 the planner’s usable horizon was 20 steps. What is the analogous long-horizon bound for the shift, and why is it not 1,152 episodes?

Answer. The bound is the drift time constant: the interval over which the environment’s statistics move by more than the tolerance. It is not 1,152 because episodes after a drift are drawn from a different distribution, so the product of per-episode survivals no longer describes the shift. Used in §L.C.1 to define the drift time constant.


Part A — The compounding law and what the floor requires

§L.A.1 — The independent law and its three regimes

The horizon failure law relates a per-episode failure probability pp to the probability of at least one failure in a horizon of nn episodes. For independent episodes,

Pfail(n)=1−(1−p)n,Psurvive(n)=(1−p)n.(A.1)P_{\text{fail}}(n) = 1 - (1-p)^n, \qquad P_{\text{survive}}(n) = (1-p)^n. \tag{A.1}

Nothing in (A.1) is subtle; everything difficult in this lesson is about which pp to substitute and whether independence holds. The law has three regimes.

Small nn. When np≪1np \ll 1, Psurvive(n)≈1−npP_{\text{survive}}(n) \approx 1 - np: survival falls linearly and the per-episode rate is a good local description. This is the regime of a single demonstration or a short trial.

Unit nn. When np≈1np \approx 1, Psurvive≈e−1=0.368P_{\text{survive}} \approx e^{-1} = 0.368. The horizon at which this happens, n≈1/pn \approx 1/p, is the mean time to failure measured in episodes. This is the characteristic horizon of the system.

Large nn. When np≫1np \gg 1, survival decays exponentially and failure is nearly certain. The horizon budget — the largest nn at which survival stays above a stated floor — is what a deployment needs, and it is this regime.

The running case sits far inside the third regime. The measured per-episode success in 4.3.2 §L.E.1 is 0.82, so p=0.18p = 0.18, and the shift is n=8×3600/25=1,152n = 8 \times 3600 / 25 = 1{,}152 episodes. The logarithm of survival is

ln⁡Psurvive=1152ln⁡0.82=1152×(−0.19845)=−228.6,\ln P_{\text{survive}} = 1152 \ln 0.82 = 1152 \times (-0.19845) = -228.6,

so Psurvive=0.821152=5.17×10−100P_{\text{survive}} = 0.82^{1152} = 5.17 \times 10^{-100}. The horizon budget at this rate is ln⁡0.90/ln⁡0.82=0.531\ln 0.90 / \ln 0.82 = 0.531 episodes: the floor is breached inside the first episode.

That number is not a prediction that the cell never completes a shift. It is a statement that a cell with no recovery and no maintenance, running at 0.82 per episode, has no shift to speak of. The observed shift does complete, and the difference between the arithmetic and the observation is the entire subject of Parts C and D: recovery and maintenance supply the survival the per-episode rate denies.

§L.A.2 — Why independence fails: the drift hazard

Equation (A.1) assumes pp is a constant that does not know what happened last episode. In a real cell the environment moves: lights cycle with the building’s HVAC, the camera’s extrinsics creep under thermal load, gripper pads wear, and the policy meets scenes further into the tails of its training distribution. When the environment moves, pp moves with it, and failures arrive in clusters rather than at a constant rate.

The correlated law replaces the constant pp with a hazard that depends on the environment’s state:

λ(t)=λ0+∑jβj dj(t),(A.2)\lambda(t) = \lambda_0 + \sum_{j} \beta_j \, d_j(t), \tag{A.2}

where λ0\lambda_0 is the baseline hazard, dj(t)d_j(t) the jj-th drift variable, and βj\beta_j its sensitivity. Survival is exp⁡!(−∫0Tλ(t) dt)\exp!\left(-\int_0^T \lambda(t)\,dt\right), and the content of the correlated law is entirely in the time dependence of λ\lambda.

For the running case the four drift variables are:

Camera calibration, d1d_1. Extrinsic parameters drift by roughly 0.1 mm per hour of thermal cycling, and pose error grows with them. The sensitivity β1\beta_1 converts millimetres of calibration error into escalation probability.

Lighting, d2d_2. Overhead lights change with the HVAC cycle, and the perception MMD rises. β2\beta_2 converts MMD units into escalation probability.

Gripper pad wear, d3d_3. Pad friction falls by about 2% per 1,000 grasps, and slip rate rises. β3\beta_3 converts friction loss into escalation probability.

Policy confidence, d4d_4. The VLA’s action distribution shifts as deployment scenes visit the tails of the training distribution. β4\beta_4 converts confidence loss into escalation probability.

The signature of the correlated law is clustering. Inter-failure times are less variable than the exponential that (A.1) implies, and a burst corresponds to the drift crossing a threshold. The measurable signature is the autocorrelation of the failure indicator: at lag kk, a positive value means a failure raises the probability of another failure kk episodes later. The lab measures this, and §L.C.1 reports the result — which is more interesting than the textbook expects, because at the running case’s drift constant the lag-1 autocorrelation is statistically indistinguishable from zero.

The practical consequence of (A.2) is that maintenance must be scheduled against the hazard integral over an interval, not against an average rate. A schedule that resets the hazard to λ0\lambda_0 truncates the integral; a schedule that does not, integrates a rising function.

Concept check. Inter-failure times are exponentially distributed with mean 10 episodes, but observed failures arrive in bursts of three to five separated by gaps of about 40 episodes. What is the diagnosis and what follows?

Answer. The exponential fit is an artefact of matching the mean. Bursts mean the hazard is time-varying, and the bursts are drift crossings. The consequence is that the independent arithmetic at p=0.10p = 0.10 reproduces the average rate and understates the tail; maintenance must trigger at burst onset, not at the average. [displaces M2]

§L.A.3 — The deployment floor and what it actually requires

The deployment floor is the minimum acceptable shift survival — the operator’s requirement. The horizon budget is the largest nn at which survival stays above it.

For the running case the operator requires Psurvive(1152)≥0.90P_{\text{survive}}(1152) \ge 0.90. Invert (A.1) at zero failures:

(1−q)1152≥0.90  ⟹  q≤1−0.901/1152=9.15×10−5.(A.3)(1-q)^{1152} \ge 0.90 \;\Longrightarrow\; q \le 1 - 0.90^{1/1152} = 9.15 \times 10^{-5}. \tag{A.3}

With the recovery rate RR folded in, the per-episode escalation probability is q=p(1−R)q = p(1-R), so (A.3) demands

R≥1−9.15×10−50.18=0.99949.R \ge 1 - \frac{9.15 \times 10^{-5}}{0.18} = 0.99949.

The composed recovery rate from 4.2.4 §L.C.3 is 0.809, and the resulting escalation rate is q=0.18×0.191=0.03438q = 0.18 \times 0.191 = 0.03438 — 376 times the rate (A.3) permits. At that rate the horizon budget is

n=ln⁡0.90ln⁡(1−0.03438)=−0.10536−0.03498=3.01 episodes,n = \frac{\ln 0.90}{\ln(1 - 0.03438)} = \frac{-0.10536}{-0.03498} = 3.01 \text{ episodes},

and the shift survival is (1−0.03438)1152=3.14×10−18(1 - 0.03438)^{1152} = 3.14 \times 10^{-18}.

This is the central quantitative result of the lesson, and it is worth stating plainly because it contradicts the natural reading of the earlier draft of this material: no maintenance schedule rescues a recovery rate of 0.809. §L.C.3 and §L.D.3 show that resets do help — 9.2 decades for the first reset, 14.7 decades by the fourth — and that even the safety-factor schedule of 5 resets, at 2.5 hours of maintenance, leaves survival at 3.4×10−213.4 \times 10^{-21}. The lever that reaches the floor is RR, not the reset frequency.

A more permissive reading of the floor gives the same verdict with a smaller shortfall. If “at least 0.90” means “at most one escalation in the shift” rather than “no escalations”, the constraint is Poisson with mean μ=qn\mu = qn:

e−μ(1+μ)≥0.90  ⟹  μ≤0.5318,e^{-\mu}(1 + \mu) \ge 0.90 \;\Longrightarrow\; \mu \le 0.5318,

so q≤4.62×10−4q \le 4.62 \times 10^{-4} and R≥0.99744R \ge 0.99744 — 74.5 times the available recovery rate rather than 376. The expected escalation count at the operating point is qn=39.6qn = 39.6 per shift, against 207.4 raw failures, so recovery removes about 81% of failures and leaves four times the floor’s allowance.

Two numbers make the required region concrete. At R=0.999R = 0.999 the shift survival is 0.813; at R=0.9995R = 0.9995 it is 0.902; at R=0.9999R = 0.9999 it is 0.980. The floor sits between 0.999 and 0.9995, and every design decision in Part D should be read as an attempt to move RR into that band, not to move the reset count.

The maintenance contribution is still real. The residual of 1.4 cm from 4.3.2 §L.C.3 is the drift seed, and a reset at drift onset returns the hazard to λ0\lambda_0. A reset is an intervention in the sense of 4.2.4 §L.C.2, and the maintenance schedule is its frequency. What the arithmetic above establishes is that the reset frequency is a second-order control relative to RR: it buys orders of magnitude in survival but cannot buy the last four.


Consolidation checkpoint 1

(a) Three capabilities. You can state the independent law and its three regimes. You can state the correlated hazard and its clustering signature. You can compute what recovery rate a stated floor demands.

(b) Transfer probe. The per-episode failure rate is 0.05 and the floor is Psurvive(100)≥0.90P_{\text{survive}}(100) \ge 0.90. Compute the shift survival and the floor status. Answer: 0.95100=5.92×10−30.95^{100} = 5.92 \times 10^{-3}, far below 0.90. The required per-episode success is 0.901/100=0.998950.90^{1/100} = 0.99895, so the required failure rate is 1.05×10−31.05 \times 10^{-3} — 47.5 times smaller than 0.05. Recovery and maintenance must supply the difference.

(c) Remediation rule. If you cannot say why the independent law overstates survival when failures are positively correlated, re-read §L.A.2 before Part B.


Part B — Long-horizon metrics and window sizing

§L.B.1 — The five metrics

Long-horizon evaluation reports five quantities, each with a bound.

Shift survival rate. The fraction of shifts that complete without an escalation. Bound: 0.90, from the operator.

Mean time between failures (MTBF). The average number of episodes between detected failures. Bound: above 10 episodes. At the running case’s raw rate MTBF is 1/0.18=5.561/0.18 = 5.56 episodes; with recovery folded in it is 1/0.03438=29.11/0.03438 = 29.1 episodes, so the bound is met only in the recovered reading. The distinction matters and must be stated whenever MTBF is quoted.

Drift time constant TdT_d. The interval over which the hazard doubles. Bound: the maintenance interval must satisfy Tm≤Td/3T_m \le T_d/3, the safety factor from 4.3.1 §L.C.1.

Recovery effectiveness decay. The composed rate at the end of the horizon divided by the rate at the start. Bound: ratio above 0.80.

Maintenance rate. Resets per shift. Bound: at most 4 for the running case.

The five are the long-horizon complement of the six metrics in 4.1.1 §L.C.1, and their conjunction is the shift acceptance.

§L.B.2 — Window sizing

Window sizing is the episode count needed to estimate a metric at a required confidence-interval resolution. It is the power arithmetic of 4.3.1 §L.C.1 applied at the shift level.

For a shift survival proportion near 0.90 at a five-point resolution with a 95% interval,

nwindow=1.962×0.90×0.100.052=138.3 shifts.(B.1)n_{\text{window}} = \frac{1.96^2 \times 0.90 \times 0.10}{0.05^2} = 138.3 \text{ shifts}. \tag{B.1}

138 shifts at 8 hours each is 1,106 hours, or 46 days of continuous operation. The window is not the deployment target: the target is 1 shift, and the window needed to measure shift survival directly is 138 times larger.

Substitution. A 138-shift window is infeasible for a capstone, so the evaluation uses an episode-level suite — the 400-episode suite of 4.3.2 §L.E.2 — to estimate the per-episode rate, and extrapolates to shift survival through the model in §L.C.3. The extrapolation is the object that must be defended, and its validity is the contract of §L.B.4.

Upper bound on the window. The window cannot exceed the drift time constant TdT_d, because episodes after a drift are drawn from a different distribution. In the running case Td=576T_d = 576 episodes, so a 400-episode suite fits inside one drift phase — but only just, and the margin is what makes stratification necessary.

Design. The 400-episode suite is the short window, inside TdT_d. The long-horizon window is the sequence of reset intervals, with shift survival extrapolated. The extrapolation’s interval is the model’s uncertainty, and the conservative choice is its lower bound.

§L.B.3 — The horizon confidence interval

The survival estimate at the horizon end carries an interval far wider than the per-episode interval. Writing s=1−ps = 1-p for the per-episode success rate and σs\sigma_s for its standard error, ln⁡Psurvive=nln⁡s\ln P_{\text{survive}} = n \ln s, and the delta method gives

Var⁡!(ln⁡Psurvive)≈n2σs2s2.(B.2)\operatorname{Var}!\left(\ln P_{\text{survive}}\right) \approx n^2 \frac{\sigma_s^2}{s^2}. \tag{B.2}

The standard deviation grows linearly in nn, so the interval on the log survival widens linearly and the interval on survival widens exponentially.

Numbers for the running case, all computed:

Suitessσs\sigma_snnsd of ln⁡P\ln P95% half-widthlog⁡10P\log_{10} Pinterval
400-episode0.820.024009.8 nats±19.1\pm 19.1 nats (±8.3\pm 8.3 decades)−34.5-34.5[10−42.8,10−26.2][10^{-42.8}, 10^{-26.2}]
shift0.820.021,15228.1 nats±55.1\pm 55.1 nats (±23.9\pm 23.9 decades)−99.3-99.3[10−123.2,10−75.4][10^{-123.2}, 10^{-75.4}]
shift0.820.031,15242.1 nats±82.6\pm 82.6 nats (±35.9\pm 35.9 decades)−99.3-99.3[10−135.2,10−63.4][10^{-135.2}, 10^{-63.4}]

The point estimate 10−9910^{-99} is not a usable statement. It is indistinguishable from 10−7510^{-75} and from 10−12310^{-123} at the same confidence, and both of those differ from it by more than any decision would tolerate. What the design uses is the lower bound, per 4.3.1 §L.C.4: a decision must be made on the pessimistic end.

Control. Stratified windows — one per drift phase — give a narrower interval than a pooled window, because within-phase variance is smaller than across-phase variance. The stratified logging of 4.3.1 §L.D.1 is the mechanism.

§L.B.4 — The extrapolation contract

The extrapolation contract is the set of assumptions under which an episode-level window predicts shift survival. It is the condition on any long-horizon evidence claim, and it has three clauses.

Hazard stationarity within the phase. The per-episode rate is constant between resets. Violation: fast drift inside a phase. Detection: the drift channel of 4.3.1 §L.D.2.

Recovery independence across episodes. Recovery effectiveness is not degraded by prior recoveries. Violation: recovery state — a consumed retry budget, an accumulated degrade tolerance. Detection: the recovery kind distribution.

Reset completeness. A reset restores the hazard to λ0\lambda_0. Violation: partial recalibration. Detection: the post-reset checks of 4.2.1 §L.B.3.

The contract is recorded in the analysis field of the 4.3.1 §L.A.2 registry, and a violated clause is a diagnosed failure mode, not a caveat to be noted in passing.

Self-explanation 2. Why is the lower bound the decision and the upper bound the optimism?

Answer. The decision is the operator’s acceptance of shift reliability, and acceptance must be conservative. The lower bound is the pessimistic estimate; the upper bound would overstate reliability if used. The lower-bound discipline of 3.1.4 §L.D.1 is the same rule applied to a different quantity. Note that at the horizon the interval is so wide that “upper bound” can exceed 1 — see the checkpoint below — which is a second reason the upper end carries no information.

Concept check. A 400-episode suite gives a per-episode success rate of 0.82 with σs=0.02\sigma_s = 0.02, and the shift survival is extrapolated as 0.8211520.82^{1152}. Is the point estimate useful?

Answer. No. By (B.2) the log interval is ±23.9\pm 23.9 decades, so 10−9910^{-99} is not distinguishable from 10−7510^{-75} or 10−12310^{-123}. What carries information is the contract of §L.B.4 plus the lower bound. [displaces M3]


Consolidation checkpoint 2

(a) Three capabilities. You can name the five metrics and their bounds. You can compute a window from an interval resolution. You can state the three contract clauses.

(b) Transfer probe. The per-episode success rate is 0.90 with σs=0.01\sigma_s = 0.01, and the horizon is 500 episodes. Compute the log interval. Answer: Var⁡(ln⁡P)=5002×0.012/0.902=25/0.81=30.9\operatorname{Var}(\ln P) = 500^2 \times 0.01^2 / 0.90^2 = 25/0.81 = 30.9, so the standard deviation is 5.56 nats and the 95% half-width is ±10.9\pm 10.9 nats, or ±4.7\pm 4.7 decades. The point estimate is 0.90500=1.32×10−230.90^{500} = 1.32 \times 10^{-23}, so the interval is [10−27.6,10−18.1][10^{-27.6}, 10^{-18.1}]. Note how much narrower this is than the running case: the interval scales with nσs/sn\sigma_s/s, and 500 episodes at σs=0.01\sigma_s = 0.01 is a far better measurement than 1,152 at 0.02.

(c) Remediation rule. If you cannot say why the window is not the shift, re-read §L.B.2 before Part C.


Part C — Drift sources and the maintenance policy

§L.C.1 — The four drift sources and the drift time constant

The drift sources are the four dj(t)d_j(t) of (A.2). Each has a detection channel and a time constant.

Camera calibration drift. Extrinsics drift about 0.1 mm per hour. Detection: the frame check of 4.2.1 §L.B.3, which compares the SLAM estimate against the kinematic prior. Time constant: 8 hours.

Lighting drift. Overhead lights track the building’s HVAC cycle. Detection: the MMD of 3.4.4 §L.D.1. Time constant: 4 hours.

Gripper pad wear. Friction falls 2% per 1,000 grasps, and a shift is roughly 1,152 grasps, so a shift costs about 2.3% of the pad’s friction. Detection: the momentum observer residual of 2.3.3 §L.E.2. Time constant: 36 hours, or about 4.5 shifts.

Policy confidence drift. The VLA’s action distribution shifts as scenes visit the training tails. Detection: the Q-shield rejection rate of 2.2.4 §L.D.3. Time constant: 16 hours, or 2 shifts.

The sensitivities βj\beta_j are the contribution shares measured in 4.3.2 §L.D.1 — perception 1.3 cm, actuation 0.5 cm, dynamics 1.1 cm, policy 0.3 cm — projected onto the escalation rate.

The drift time constant TdT_d is the smallest of the four, which is 4 hours (lighting), not the 8-hour camera constant and not the 16-hour policy constant. The safety factor of 4.3.1 §L.C.1 then gives Tm≤Td/3=1.33T_m \le T_d/3 = 1.33 hours. At a 30-minute reset cost that is 6 intervals, 5 resets, and 2.5 hours of maintenance. The running case’s 4 resets per shift give a 1.60-hour interval — a looser interval than the safety factor allows, by 20%. §L.D.3 works the cost of both.

The choice of the smallest constant is not a conservative flourish. Each drift has its own detection channel, and the channel with the shortest constant is the one that fires first; scheduling to any larger constant leaves the lighting channel’s drift unaddressed between resets.

§L.C.2 — The maintenance policy: three thresholds

The maintenance policy is the schedule of resets. It has three thresholds, each a decision.

Monitor threshold. The drift-channel statistic — MMD for lighting, calibration disagreement for the camera, observer residual for wear, Q-shield rejection rate for policy — is compared against the baseline upper bound. A statistic above the bound is a drift detection.

Reset threshold. A statistic sustained above the bound for a dwell of three consecutive monitor windows triggers a reset. The dwell is the false-alarm control, the hysteresis of 4.2.4 §L.B.3.

Escalation threshold. A reset that fails — the post-reset statistic still above the bound — triggers escalation to the operator, per 4.2.4 §L.C.2.

The three are the reactive half. The proactive half is a reset at Td/3T_d/3 regardless of the statistic. The trade is direct: proactive resets cost a fixed amount per interval and eliminate the drift entirely; reactive resets cost only when drift is detected but carry the risk of drift between detection and reset.

For the running case a reset is a recalibration package — camera extrinsic re-check, lighting baseline re-capture, gripper friction re-identification, Q-shield recalibration — costing 30 minutes.

§L.C.3 — The recovery compounding

Recovery compounding is the decay of recovery effectiveness across a horizon. It is the composed rate of 4.2.4 §L.C.3 nested inside the horizon law, and the nesting is where a common error lives.

The composed recovery rate is R=0.809R = 0.809: the probability that a detected failure is recovered without escalation. It is not a survival probability, and it does not raise to a power as one. What raises to a power is the per-episode escalation probability,

q=p (1−R)=0.18×0.191=0.03438,(C.1)q = p\,(1 - R) = 0.18 \times 0.191 = 0.03438, \tag{C.1}

and the survival of a kk-episode segment is (1−q)k(1-q)^k.

The distinction is worth a factor. At k=48k = 48 the correct segment survival is (1−q)48=0.1865(1-q)^{48} = 0.1865; the tempting R48=3.8×10−5R^{48} = 3.8 \times 10^{-5} is off by a factor of 4,900 and points at the wrong design conclusion. The temptation is the substitution of the recovery’s success probability for the shift’s survival probability — two quantities with the same units and no relation.

Recovery effectiveness does decay, and for three reasons.

Retry budget. The two-retry bound is consumed by earlier failures, so late failures have fewer retries.

Degrade tolerance. Coarser tolerances accumulate: the kk-th degrade is kk times coarser than the first.

Abort state. An abort reaches the safe state and the next episode starts from it, per step 4 of 4.2.4 §L.C.2, which is itself a reset.

The compounding law is therefore (1−q)k(1-q)^k nested inside the drift segments, with qq rising across segments as RR falls. The measured decay is the subject of §L.D.1.

For the running case at Td=4T_d = 4 hours and the 1.33-hour interval, a segment is 192 episodes and the segment survival is (1−q)192=1.2×10−3(1-q)^{192} = 1.2 \times 10^{-3} — still catastrophic, which is why §L.A.3’s verdict stands.

Self-explanation 3. Why is the compounding per-segment rather than per-episode, and why is RR not the quantity that compounds?

Answer. The maintenance reset restores the recovery’s state — recalibration resets the retry budget and the degrade tolerance — so a segment is the interval between resets, and the compounding is within it. And RR is the probability of recovering a failure, not of surviving an episode; the episode survives if it does not fail, or if it fails and is recovered, so the escalation probability is p(1−R)p(1-R) and that is what compounds.

Concept check. The recovery rate is 0.90, the per-episode failure rate is 0.18, and the segment is 6 episodes. Compute the segment survival and the segment failure probability.

Answer. q=0.18×0.10=0.018q = 0.18 \times 0.10 = 0.018, so the segment survival is (1−0.018)6=0.897(1 - 0.018)^6 = 0.897 and the segment failure is 0.103. Note that 0.906=0.5310.90^6 = 0.531 is the wrong answer and is wrong in the direction that makes the system look worse than it is. [displaces M5]


Consolidation checkpoint 3

(a) Three capabilities. You can name the four drift sources, their channels, and their constants. You can state the three maintenance thresholds. You can compute a segment survival from pp and RR.

(b) Transfer probe. The recovery rate is 0.85, the per-episode failure rate is 0.18, and the segment is 10 episodes. Compute the segment survival. Answer: q=0.18×0.15=0.027q = 0.18 \times 0.15 = 0.027, so the segment survival is (1−0.027)10=0.761(1-0.027)^{10} = 0.761 and the segment failure is 0.239. For contrast, 0.8510=0.1970.85^{10} = 0.197 — the wrong formula again, and again pessimistic.

(c) Remediation rule. If you cannot say why q=p(1−R)q = p(1-R) rather than RR, re-read §L.C.3 before Part D.


Part D — Recovery across horizons, degradation modes, schedule design

§L.D.1 — The recovery effectiveness decay

Recovery effectiveness decay is the rate at which the composed rate falls across segments. Its measurement is the interval of 4.3.1 §L.C.4 at each segment index.

The form is Rs=(1−qs)ksR_s = (1 - q_s)^{k_s} with ksk_s the segment length and qsq_s the segment’s escalation rate. Decay means qsq_s rises with segment index as recovery state accumulates; the signature is the recovery kind distribution shifting toward abort and escalate.

Measurement. At Td/3T_d/3 the running case has 5 resets per shift and segments of 192 episodes. A 400-episode suite covers roughly two segments, which is not enough to fit a trend. The interval on a per-segment survival at 192 episodes and σs=0.02\sigma_s = 0.02 is, by (B.2), ±4.0\pm 4.0 decades — wide enough that a decay of the size the mechanism predicts is not separable from noise at this sample size. This is a real limitation, and the honest response is to report the decay as unmeasured rather than to fit a trend to two points.

Consequence. A detected decay re-runs the stopping rule of 4.3.2 §L.C.3 on the residual: the residual is no longer stationary, so the diagnosis must be reopened.

§L.D.2 — The four degradation modes

Degradation modes are the ways a system fails over a horizon. Each has a signature and a detector.

Cumulative wear. Pad friction falls and slip rate rises. Signature: monotone rise in slip rate across shifts. Detector: the momentum observer residual of 2.3.3 §L.E.2.

Calibration drift. Extrinsics move and pose error grows. Signature: monotone rise in the frame-check disagreement of 4.2.1 §L.B.3. Detector: the frame check.

Policy distribution shift. Deployment scenes visit training tails and confidence falls. Signature: monotone rise in Q-shield rejection rate. Detector: the rejection rate.

Recovery state. Retry budget consumed and degrade tolerance accumulated. Signature: the recovery kind distribution shifts toward abort and escalate. Detector: the kind distribution.

The first three are environment- or hardware-driven and respond to resets. The fourth is the recovery’s own degradation, and it responds to resets too — which is why the reset schedule is the answer to a mode that is otherwise self-reinforcing. The four are the horizon entries of the 3.3.4 §L.E.3 register.

§L.D.3 — The maintenance schedule and its trade

The maintenance schedule is the set of reset times across a shift, composed from the proactive and reactive halves of §L.C.2.

Proactive at Td/3T_d/3. Td=4T_d = 4 h gives a 1.33-hour interval: 6 intervals, 5 resets per shift, at 30 minutes each, is 2.5 hours or 31% of the shift.

Reactive. Reset on detection with a three-window dwell. Cost is 30 minutes times the detection count, typically three or four per shift. Risk is the drift between detection and reset.

Composed. Proactive at a 1.60-hour interval — 4 resets, 2.0 hours — plus reactive on detection. This is the running case’s schedule, and it costs 25% of the shift for the proactive half.

The trade is not subtle. Proactive cost is linear in frequency; reactive cost is the detection count, and its risk is the unmonitored drift. The optimum is the expected-information-gain arithmetic of 4.3.1 §L.C.2: reset when the expected reduction in hazard integral exceeds the reset’s cost.

What the schedule cannot do is set RR. §L.A.3 established that the floor needs R≥0.9995R \ge 0.9995 and the composed rate is 0.809, so the schedule buys survival orders of magnitude and stops roughly 17 orders short. The running case’s schedule is therefore a drift control, not a floor-compliance mechanism, and the register in §L.E.3 must say so.


Consolidation checkpoint 4

(a) Three capabilities. You can state the decay form and why it is hard to measure. You can name the four degradation modes and their signatures. You can state the composed schedule and its cost.

(b) Transfer probe. The proactive reset costs 15 minutes and Td=16T_d = 16 hours. Compute the proactive frequency and per-shift cost. Answer: Td/3=5.33T_d/3 = 5.33 hours, which is more than half a shift, so a shift contains at most one proactive reset; the cost is 15 minutes. If instead the schedule uses Td/2=8T_d/2 = 8 hours, it is one reset per shift at 15 minutes. The point of the probe is that a long TdT_d makes the proactive half cheap, and the running case’s Td=4T_d = 4 hours is what makes it expensive.

(c) Remediation rule. If you cannot say why the recovery kind distribution is the signature of the fourth mode, re-read §L.D.2 before Part E.


Part E — The running case

§L.E.1 — The 8-hour shift plan

The plan assembles the evidence chain of 4.3.2 §L.E.1 and the recovery budget of 4.2.4 §L.E.2.

Target. 1,152 episodes at 25 seconds, shift survival floor 0.90, at most 4 resets per shift.

Rates. Per-episode failure 0.18; composed recovery 0.809; escalation q=0.03438q = 0.03438.

Segments. Four proactive resets cut the shift into five intervals, and 1,152 does not divide by 5: 1152/5=230.41152/5 = 230.4. A schedule of five equal segments of 230 episodes silently drops two episodes, so the last segment takes the remainder and the plan reports 230×4+232230 \times 4 + 232. The interval is then 1.60 hours, not the 2 hours a five-into-eight division would suggest.

This is where the two constraints meet and bind. The maintenance budget of 4 resets caps the interval from below at 1.60 hours. The safety factor Tm≤Td/3T_m \le T_d/3 requires 1.33 hours, which needs 5 resets and costs 2.5 hours. The running case’s 4 resets therefore violate the safety factor by 20% to stay inside the cost bound, and §L.D.3 is the account of that trade.

Schedule. 4 proactive resets at a 1.60-hour interval, plus reactive resets on detection. Cost 2.0 hours per shift; risk is the drift inside the interval.

Numbers.

ComponentValueSource
Shift episodes1,1528 h / 25 s
Per-episode failure rate0.184.3.2 §L.E.1
Recovery rate0.8094.2.4 §L.C.3
Escalation rate qq0.03438p(1−R)p(1-R)
Expected escalations per shift39.6q×1152q \times 1152
Raw failures per shift207.4p×1152p \times 1152
Shift survival, constant hazard3.14×10−183.14 \times 10^{-18}(1−q)1152(1-q)^{1152}
Shift survival, drift, no reset1.83×10−361.83 \times 10^{-36}linear hazard
Shift survival, drift, 4 resets8.57×10−228.57 \times 10^{-22}5 segments, 230 + 232
Required RR for floor (zero-failure)0.99949(A.3)
Required RR for floor (at most one)0.99744Poisson
Maintenance cost2.0 h4 × 30 min

The two survival rows are the lesson’s punchline. Drift costs 18 orders of magnitude; four resets recover 14 of them; the floor is still 21 orders away. The schedule is doing exactly what it was designed to do, and it is not the constraint that matters.

§L.E.2 — The horizon evidence

Metrics. The five of §L.B.1 with their bounds.

Window. The 400-episode suite, plus the stratified per-phase windows.

Interval. At 400 episodes and σs=0.02\sigma_s = 0.02 the log interval is ±8.3\pm 8.3 decades; extrapolated to the shift it is ±23.9\pm 23.9 decades. The extrapolation contract, not the interval, is what makes the statement usable.

Evidence failure. The 1.4 cm residual of 4.3.2 §L.E.2 is the drift seed, and the reset is the residual’s reset. The horizon evidence is the pre-reset and post-reset comparison.

Conclusion. The shift survival lower bound is the decision, per 4.3.1 §L.C.4. At the running case’s numbers the lower bound is far below the floor, so the honest conclusion of the horizon evidence is a refusal: the cell does not meet its reliability requirement at the current recovery rate, and the register must record that.

§L.E.3 — The register entries

FailureProbabilityConsequenceMitigationResidual
Drift unmonitored inside the reset intervalModerateClustered failures4 proactive resetsDrift within the 1.60-hour interval
Recovery effectiveness decayHighEscalation rate rises across segmentsReset scheduleSegment length
Policy confidence driftHighQ-shield rejection rate risesRecalibrationRecalibration cost
Gripper wearModerateSlip rate risesFriction re-identificationWear between resets
Extrapolation contract violationModeratePrediction invalidStratified windowsThe three contract clauses
Recovery rate below the floor requirementHighShift survival 10−1810^{-18}, floor 0.90None available at R=0.809R = 0.809The 0.19 unrecovered fraction

The last row is the one the arithmetic forces and the earlier draft of this material omitted. A register that lists drift and wear but not the recovery-rate shortfall is a register that has not read its own §L.A.3.

Self-explanation 2. Why is within-interval drift the residual even with reactive resets?

Answer. A reactive reset fires after the statistic crosses, so drift inside the detection latency is unmonitored by construction. The monitor’s window is the detection latency, and it is not zero. The 4.3.1 §L.D.2 drift channel sets that window.

Worked-example ladder.

WE-L.E.a (fully worked). The shift is 1,152 episodes with 4 resets. Compute the segment survival and the shift survival under the constant hazard.

Solution. Four resets give five intervals, and 1,152 divides by 5 as 230.4, so the segments are four of 230 episodes and one of 232. The escalation rate is q=0.03438q = 0.03438, so under the constant hazard the shift survives with probability (1−q)1152=3.14×10−18(1-q)^{1152} = 3.14 \times 10^{-18} and a 230-episode segment survives with (1−q)230=3.20×10−4(1-q)^{230} = 3.20 \times 10^{-4}. Under the constant hazard the resets do not change the product: five segments totalling 1,152 episodes is 1,152 episodes once. The resets matter only because the hazard is not constant, which is the point of §L.A.2.

WE-L.E.b (completion). Now include the drift: λ(t)=q(1+t/Td)\lambda(t) = q(1 + t/T_d) with Td=576T_d = 576 episodes, phase reset to zero at each reset. Compute the shift survival at 0, 1, 2, 3, 4, 5, and 7 resets.

Completion.

SegmentsResetsInterval (h)Cost (h)Shift survivalvs 0 resets
108.000.01.83×10−361.83 \times 10^{-36}1×
214.000.53.15×10−273.15 \times 10^{-27}1.7×1091.7 \times 10^{9}×
322.671.03.37×10−243.37 \times 10^{-24}1.8×10121.8 \times 10^{12}×
432.001.51.08×10−221.08 \times 10^{-22}5.9×10135.9 \times 10^{13}×
541.602.08.57×10−228.57 \times 10^{-22}4.7×10144.7 \times 10^{14}×
651.332.53.40×10−213.40 \times 10^{-21}1.9×10151.9 \times 10^{15}×
871.003.51.90×10−201.90 \times 10^{-20}1.0×10161.0 \times 10^{16}×

Every reset buys survival, and the returns diminish steeply: the first reset is worth 9.24 decades, the fourth is worth 0.90, and the eighth adds 0.75 on top of the fifth. The floor at 0.90 is not reachable at any of these costs. The table is the quantitative form of §L.D.3’s trade, and it also shows the safety-factor bind: the budget allows 4 resets, while Tm≤Td/3T_m \le T_d/3 requires 5.

WE-L.E.c (faded). Write the register entry for the recovery-rate shortfall.

Faded step. Failure: the composed recovery rate is 0.809 and the floor requires 0.9995. Probability: certain — it is arithmetic. Consequence: shift survival 3.14×10−183.14 \times 10^{-18} against a floor of 0.90. Mitigation: none available by scheduling; the recovery must be improved. Residual: the 0.191 unrecovered fraction.

Completion. The residual’s acceptance is the operator’s, and it is not acceptable at these numbers. The register’s purpose is to make that visible before a shift rather than after.

WE-L.E.d (contrastive near-miss). A team reports a shift survival point estimate of 0.999 as the acceptance, and a lower bound of 0.97 which is above the 0.90 floor. Why it fails: the point estimate is not the decision, and the interval is wide enough that the lower bound at 0.97 could come from a suite whose true rate is far worse; more importantly, the estimate must be checked against the compounding law rather than against a floor alone. If the suite’s per-episode rate is 0.82 and the shift is 1,152 episodes, a shift survival of 0.999 is arithmetically impossible without a recovery rate near 0.9995, and the team’s number therefore contradicts its own inputs. The fix: recompute from pp, RR, and the segment structure, and report the lower bound of that computation.

Comparative table: horizon evidence items.

ItemValueBoundStatus
Shift survival lower bound<10−18< 10^{-18}≥0.90\ge 0.90Fail
MTBF (with recovery)29.1 episodes>10> 10Pass
TdT_d4 h—Reported
Recovery effectiveness decayNot measurable at 2 segments>0.80> 0.80Unmeasured
Maintenance rate4 per shift≤4\le 4Pass
Maintenance cost2.0 h≤2\le 2 hPass

Design principle P59.1. The floor is a constraint on the recovery rate, and the maintenance schedule is a constraint on the drift integral; a schedule that meets its cost bound and leaves the recovery rate short has met the cheaper constraint and failed the one that decides.


Consolidation checkpoint 5

(a) Three capabilities. You can state the shift plan’s numbers. You can read the horizon evidence against its bounds. You can write the register entries, including the recovery shortfall.

(b) Transfer probe. Resets are 4 per shift at 30 minutes and the maintenance budget is 1.5 hours. Is the schedule feasible? Answer: no — 4×30=1204 \times 30 = 120 minutes exceeds 90. The fix is 3 resets at a 2-hour interval, costing 1.5 hours, and the price is a longer segment and a larger drift integral within it. Note what the fix does not do: it does not change qq, so the shift survival falls from 8.57×10−228.57 \times 10^{-22} to 3.37×10−243.37 \times 10^{-24} — a factor of 254, against a floor that is 21 orders away.

(c) Remediation rule. If you cannot say why the segment count must divide the episode count, re-read §L.E.1 before the lab.


Failure taxonomy

SymptomCauses rankedDiscriminating testFix
Shift survival below floorRecovery rate short; resets too few; drift fastCompute required RR from (A.3) and compareRaise RR; resets alone cannot close it
Failures clusterDrift correlated; recovery stateLag-kk autocorrelation, compared against the ±0.03\pm 0.03 noise floorReset at drift onset
Recovery decays across segmentsRecovery state; drift; segment too longRecovery kind distributionShorten segments; reset the retry budget
Extrapolation invalidContract clause violated; window too shortStratified windowsRe-examine the clause
Maintenance cost over budgetToo many resets; reset too expensiveCost arithmeticFewer resets, longer segments, accept the drift
Drift unmonitoredMonitor window too wide; threshold too looseMonitor latencyTighten the window
Autocorrelation reads zero at a real driftDrift too slow relative to the failure rateSweep TdT_d and compare to noiseReport the test’s power, not the point estimate

The last row is not in the earlier draft and belongs there: §L.C.1 promises that clustering is the correlated law’s signature, and the lab finds the signature absent at the operating point. A taxonomy that lists only the failures the lab exercised would miss the failure of the lab’s own instrument.


The lab

Task. Implement the horizon failure law, the drift hazard, and the segment compounding for the running case’s 8-hour shift. Compute shift survival at several reset schedules under both the constant and the linear-drift hazard, and report the recovery rate the floor requires.

Conditions. Python 3.10+, NumPy. Per-episode failure 0.18; recovery 0.809; Td=4T_d = 4 h =576= 576 episodes; four drift variables with constants 8, 4, 36, and 16 hours. Seed every RNG.

Controls. (i) zero resets; (ii) 1 reset; (iii) 2 resets; (iv) 3 resets; (v) the constant hazard as a reference, under which every schedule gives the same product.

Metrics.

MetricDefinitionTarget
Shift survivalProduct over segments of the discrete drift hazardReport per schedule
Segment survival(1−q)k(1-q)^k under the constant hazardReport
Required RRFrom (A.3)Report
MTBF1/q1/q>10> 10
Maintenance costResets × 30 min≤2\le 2 h
Lag-1 autocorrelationOf the failure indicatorReport with its noise floor

Pre-registered hypotheses.

H1. The 4-reset schedule meets the 0.90 floor, because the per-segment failure is reset before it compounds.

H2. Zero resets gives the independent-law survival 10−9910^{-99} and the drift law a higher survival, because the drift saturates the hazard.

H3. The 2-reset schedule sits below the floor under the independent law and above it under the correlated law, because drift within the interval is the effective rate.

H4. The drift law’s lag-1 autocorrelation is positive and the independent law’s is zero.

Break-it sequence.

  1. Substitute RkR^k for the segment survival. The 48-episode segment reports 3.8×10−53.8 \times 10^{-5} instead of 0.1870.187, and the shift survival becomes a product of numbers that are not survival probabilities. Silent: the number looks like a low probability, and low is what the floor expects.
  2. Drop the recovery rate from the per-episode rate. Set q=pq = p and the shift survival is 10−9910^{-99}, which is 81 orders worse than the model’s own answer. Silent: a very small number looks like a conservative bound.
  3. Report the point estimate instead of the lower bound. The shift survival is 3.14×10−183.14 \times 10^{-18} and its 95% interval spans ±24\pm 24 decades. Silent: the point estimate is the plausible one.

Acceptance criteria.

CriterionPass conditionMeasured
H1 supported4-reset survival ≥0.90\ge 0.90Refuted — 1.08×10−221.08 \times 10^{-22}
H2 supportedDrift survival above the independent lawRefuted with the sign reversed — 1.83×10−361.83 \times 10^{-36} vs 3.14×10−183.14 \times 10^{-18}
H3 supported2-reset between the two lawsRefuted — both schedules fail
H4 supportedLag-1 autocorrelation positive and outside noiseRefuted at Td=4T_d = 4 h — +0.010±0.034+0.010 \pm 0.034
Break-it 1 detectedThe RkR^k substitution is caughtCaught by the segment-survival check
Break-it 2 detectedThe dropped recovery rate is caughtCaught by the required-RR computation
Break-it 3 detectedThe point-estimate report is caughtCaught by the interval width

What good looks like. A good artefact has five properties. (1) The independent and drift laws are computed and compared. (2) The segment survival uses (1−q)k(1-q)^k, not RkR^k. (3) The required recovery rate is computed from the floor and compared against 0.809. (4) The reset schedules are tabulated with their costs. (5) The autocorrelation is reported with its noise floor, and a null result is reported as a null.

A runnable core, seeded and annotated:

# lab_4_3_3.py — horizon failure law, drift hazard, reset schedules, horizon CI
import numpy as np
from math import log, log10, exp

# ---- running-case constants -------------------------------------------
$P_FAIL$ = 0.18          # per-episode failure rate; 4.3.2 §L.E.1 success 0.82
$R_REC$ = 0.809          # composed recovery rate; 4.2.4 §L.C.3
CYCLE_S = 25
SHIFT_H = 8.0
$N_SHIFT$ = int(SHIFT_H * 3600 / CYCLE_S)      # 1152 episodes
EP_PER_H = $N_SHIFT$ / SHIFT_H                 # 144
FLOOR = 0.90
RESET_MIN = 30

# four drift time constants, in hours (camera, lighting, wear, policy)
$T_DRIFT_H$ = dict(camera=8.0, lighting=4.0, wear=36.0, policy=16.0)
$T_D_H$ = $min(T_DRIFT_H$.values())              # 4.0 h
$T_D$ = $T_D_H$ * EP_PER_H                       # 576 episodes

Q = $P_FAIL$ * (1.0 - $R_REC$)                   # per-episode escalation rate

# ---- the two laws -----------------------------------------------------
def surv_const(q, n):
    """Independent episodes: no resets can help, the product is fixed."""
    return (1.0 - q) ** n

def $segment_lengths(n_resets$, $n=N_SHIFT$):
    """$n_resets$ cut points give $n_resets+1$ intervals; the last absorbs the remainder."""
    m = $n_resets$ + 1
    base = n // m
    return [base] * (m - 1) + [n - base * (m - 1)]

def $surv_drift(n_resets$, q=Q, $t_d=T_D$, $n=N_SHIFT$):
    """Each interval restarts the hazard at lambda_0 with drift phase zero."""
    ks = $segment_lengths(n_resets$, n)
    assert sum(ks) == n, "segments must tile the shift"
    return float(np.prod([np.prod(1.0 - q * (1.0 + np.arange(k) / $t_d$)) for k in ks]))

def horizon_budget(q, floor=FLOOR):
    return log(floor) / log(1.0 - q)

def required_rate(floor=FLOOR, $n=N_SHIFT$):
    """Per-episode escalation rate that meets the floor with zero failures."""
    return 1.0 - floor ** (1.0 / n)

def $lag_autocorr(t_d=T_D$, lag=1, $n=N_SHIFT$, q=Q, seed=0):
    rng = np.random.default_rng(seed)
    f = (rng.random(n) < q * (1.0 + np.arange(n) / $t_d$)).astype(float)
    return 0.0 if f.std() == 0 else float(np.corrcoef(f[:-lag], f[lag:])[0, 1])

def log_surv_ci(p, sigma, n, z=1.96):
    """Delta method on ln $P_survive(n$) = n ln(1 - p); p is the failure rate."""
    sd = n * sigma / (1.0 - p)
    return sd, z * sd

# ---- canaries ---------------------------------------------------------
print("== canaries ==")
c1 = surv_drift(4, $t_d=1e12$)
print("C1 $T_d-$>inf must equal the constant law : %.15e vs %.15e  rel %.1e"
      % (c1, surv_const(Q, $N_SHIFT$), abs(c1 - surv_const(Q, $N_SHIFT$)) / surv_const(Q, $N_SHIFT$)))
print("C2 q=0 must give 1                      : %.15f" % surv_drift(4, q=0.0))
print("C3 alternating pattern lag-1 must be -1 : %.4f"
      % np.corrcoef([0, 1, 0, 1, 0, 1][:-1], [0, 1, 0, 1, 0, 1][1:])[0, 1])
rng = np.random.default_rng(7)
nmc, sig = 400, np.$sqrt(P_FAIL$ * (1 - $P_FAIL$) / 400)
sd_mc = np.std(nmc * np.log(1 - rng.binomial(nmc, $P_FAIL$, 20000) / nmc))
print("C4 delta method vs Monte Carlo at n=400 : %.4f vs %.4f" % ($log_surv_ci(P_FAIL$, sig, nmc)[0], sd_mc))

# ---- the laws ---------------------------------------------------------
print("\n== laws at n = %d, q = %.5f ==" % ($N_SHIFT$, Q))
print("independent, raw p = %.2f        : %.4e  (10^%.1f)" % ($P_FAIL$, $surv_const(P_FAIL$, $N_SHIFT$), $N_SHIFT$ * log10(1 - $P_FAIL$)))
print("independent, with recovery       : %.4e  (10^%.1f)" % (surv_const(Q, $N_SHIFT$), $N_SHIFT$ * log10(1 - Q)))
print("drift hazard, no reset           : %.4e  (10^%.1f)" % (surv_drift(0), log10(surv_drift(0))))
print("MTBF raw %.2f episodes, with recovery %.1f episodes" % (1 / $P_FAIL$, 1 / Q))
print("horizon budget raw %.2f episodes, with recovery %.2f episodes"
      % ($horizon_budget(P_FAIL$), horizon_budget(Q)))

# ---- what the floor requires -----------------------------------------
qr = required_rate()
print("\n== floor %.2f over %d episodes ==" % (FLOOR, $N_SHIFT$))
print("zero-failure reading : q <= %.4e  ->  R >= %.5f   (have %.4e, short %.0fx)"
      % (qr, 1 - qr / $P_FAIL$, Q, Q / qr))
lo, hi = 0.0, 20.0
for _ in range(300):
    mid = (lo + hi) / 2
    lo, hi = (mid, hi) if exp(-mid) * (1 + mid) > FLOOR else (lo, mid)
lam = (lo + hi) / 2
print("at-most-one reading  : lam <= %.4f escalations/shift  ->  q <= %.4e, R >= %.5f (short %.1fx)"
      % (lam, lam / $N_SHIFT$, 1 - (lam / $N_SHIFT$) / $P_FAIL$, Q / (lam / $N_SHIFT$)))
print("expected escalations per shift: %.1f (raw failures %.1f)" % (Q * $N_SHIFT$, $P_FAIL$ * $N_SHIFT$))

# ---- reset schedules --------------------------------------------------
print("\n== reset schedules, $T_d$ = %.0f h = %.0f episodes ==" % ($T_D_H$, $T_D$))
print("resets segs interval_h cost_h  drift_surv   const_surv  vs_zero")
base = surv_drift(0)
for nr in (0, 1, 2, 3, 4, 5, 7):
    ks = segment_lengths(nr)
    s = surv_drift(nr)
    print("%6d %4d %10.2f %6.1f  %.3e  %.3e  %8.1fx"
          % (nr, len(ks), $N_SHIFT$ / len(ks) / EP_PER_H, nr * RESET_MIN / 60.0,
             s, surv_const(Q, $N_SHIFT$), s / base))

# ---- autocorrelation --------------------------------------------------
print("\n== lag-1 autocorrelation of the failure indicator (30 seeds) ==")
for td_h in (8.0, 4.0, 2.0, 1.0, 0.5, 0.25):
    v = np.array([lag_autocorr(td_h * EP_PER_H, seed=s) for s in range(30)])
    print("$T_d$ = %4.2f h (%5.0f ep) : %+.3f +- %.3f" % (td_h, td_h * EP_PER_H, v.mean(), v.std()))

# ---- horizon CI -------------------------------------------------------
print("\n== CI on ln $P_survive$, delta method ==")
for p, sigma, n, tag in [($P_FAIL$, 0.02, 400, "400-episode suite"),
                         ($P_FAIL$, 0.0192, 1152, "shift, binomial SE"),
                         ($P_FAIL$, 0.03, 1152, "shift, sigma_p = 0.03"),
                         (0.10, 0.01, 500, "checkpoint 2")]:
    sd, half = log_surv_ci(p, sigma, n)
    lp = n * log10(1 - p)
    print("%-19s sd %5.1f nats  95%% +-%.1f nats (+-%.1f decades)  logP %.1f  CI [%.1f, %.1f]"
          % (tag, sd, half, half / log(10), lp, lp - half / log(10), lp + half / log(10)))

Measured output (this machine, NumPy, seeds as above):

== canaries ==
C1 $T_d-$>inf must equal the constant law : 3.139082114546570e-18 vs 3.139082129314720e-18  rel 4.7e-09
C2 q=0 must give 1                      : 1.000000000000000
C3 alternating pattern lag-1 must be -1 : -1.0000
C4 delta method vs Monte Carlo at n=400 : 9.3704 vs 9.3891

== laws at n = 1152, q = 0.03438 ==
independent, raw p = 0.18        : 5.1708e-100  (10^-99.3)
independent, with recovery       : 3.1391e-18  (10^-17.5)
drift hazard, no reset           : 1.8253e-36  (10^-35.7)
MTBF raw 5.56 episodes, with recovery 29.1 episodes
horizon budget raw 0.53 episodes, with recovery 3.01 episodes

== floor 0.90 over 1152 episodes ==
zero-failure reading : q <= 9.1455e-05  ->  R >= 0.99949   (have 3.4380e-02, short 376x)
at-most-one reading  : lam <= 0.5318 escalations/shift  ->  q <= 4.6164e-04, R >= 0.99744 (short 74.5x)
expected escalations per shift: 39.6 (raw failures 207.4)

== reset schedules, $T_d$ = 4 h = 576 episodes ==
resets segs interval_h cost_h  drift_surv   const_surv  vs_zero
     0    1       8.00    0.0  1.825e-36  3.139e-18       1.0x
     1    2       4.00    0.5  3.151e-27  3.139e-18  1726250000.1x
     2    3       2.67    1.0  3.367e-24  3.139e-18  1844666440738.4x
     3    4       2.00    1.5  1.078e-22  3.139e-18  59042696888076.5x
     4    5       1.60    2.0  8.565e-22  3.139e-18  469215817542328.0x
     5    6       1.33    2.5  3.402e-21  3.139e-18  1863726826010834.5x
     7    8       1.00    3.5  1.901e-20  3.139e-18  10417059631620288.0x

== lag-1 autocorrelation of the failure indicator (30 seeds) ==
$T_d$ = 8.00 h ( 1152 ep) : +0.003 +- 0.032
$T_d$ = 4.00 h (  576 ep) : +0.010 +- 0.034
$T_d$ = 2.00 h (  288 ep) : +0.022 +- 0.036
$T_d$ = 1.00 h (  144 ep) : +0.038 +- 0.035
$T_d$ = 0.50 h (   72 ep) : +0.118 +- 0.036
$T_d$ = 0.25 h (   36 ep) : +0.379 +- 0.035

== CI on ln $P_survive$, delta method ==
400-episode suite   sd   9.8 nats  95% +-19.1 nats (+-8.3 decades)  logP -34.5  CI [-42.8, -26.2]
shift, binomial SE  sd  27.0 nats  95% +-52.9 nats (+-23.0 decades)  logP -99.3  CI [-122.2, -76.3]
shift, sigma_p = 0.03 sd  42.1 nats  95% +-82.6 nats (+-35.9 decades)  logP -99.3  CI [-135.2, -63.4]
checkpoint 2        sd   5.6 nats  95% +-10.9 nats (+-4.7 decades)  logP -22.9  CI [-27.6, -18.1]

Reading the result. Three of four hypotheses are refuted, and each refutation has a mechanism worth keeping.

H1 fails because a reset cannot lower the escalation rate. Under the constant hazard the product is invariant to segmentation — six segments of 192 episodes is 1,152 episodes — and under the drift hazard the resets buy 14 orders of magnitude out of the 18 the drift costs, which is not enough because the starting point was already 17 orders below the floor.

H2 fails with the sign reversed. Drift does not saturate the hazard and improve survival; it makes survival worse, by 18 orders, because the hazard integral over a rising function exceeds the integral over its mean. The “saturation” intuition comes from a bounded hazard, and the linear hazard of (A.2) is not bounded within a shift.

H3 fails for the same reason as H1: both schedules are far below the floor, so there is no between.

H4 fails at the operating point and passes elsewhere, which is the most useful result. The lag-1 autocorrelation is +0.010±0.034+0.010 \pm 0.034 at Td=4T_d = 4 hours — indistinguishable from zero — and only becomes visible below about Td=0.5T_d = 0.5 hours, where it reads +0.118±0.036+0.118 \pm 0.036. The reason is that a failure rate of 0.034 per episode is one failure every 29 episodes, so a drift constant of 576 episodes is 20 mean inter-failure times; within one inter-failure gap the hazard barely moves, and the clustering the law predicts is spread over a window the sample cannot resolve. The correlated law is real; its lag-1 signature has no power at this operating point. The honest report is the power curve, not the point estimate.


Cheat sheet

Term or resultStatementRole
Horizon failure lawPsurvive(n)=(1−q)nP_{\text{survive}}(n) = (1-q)^n4.3.3, Eq. (A.1)
Drift hazardλ(t)=λ0+∑jβjdj(t)\lambda(t) = \lambda_0 + \sum_j \beta_j d_j(t)4.3.3, Eq. (A.2)
Escalation rateq=p(1−R)q = p(1-R)4.3.3, Eq. (C.1)
Required rate for a floorq≤1−F1/nq \le 1 - F^{1/n}4.3.3, Eq. (A.3)
Horizon budgetn=ln⁡F/ln⁡(1−q)n = \ln F / \ln(1-q)4.3.3, §L.A.3
Deployment floorMinimum acceptable shift survival4.3.3, §L.A.3
Five metricsSurvival, MTBF, TdT_d, decay, maintenance rate4.3.3, §L.B.1
Window sizingn=z2p(1−p)/ϵ2n = z^2 p(1-p)/\epsilon^24.3.3, Eq. (B.1)
Horizon intervalVar⁡(ln⁡P)=n2σs2/s2\operatorname{Var}(\ln P) = n^2\sigma_s^2/s^24.3.3, Eq. (B.2)
Extrapolation contractStationarity, recovery independence, reset completeness4.3.3, §L.B.4
Four drift sourcesCamera 8 h, lighting 4 h, wear 36 h, policy 16 h4.3.3, §L.C.1
Drift constantTd=min⁡jTj=4T_d = \min_j T_j = 4 h4.3.3, §L.C.1
Three thresholdsMonitor, reset, escalate4.3.3, §L.C.2
Segment survival(1−q)k(1-q)^{k}, not RkR^{k}4.3.3, §L.C.3
Four degradation modesWear, calibration, distribution, recovery state4.3.3, §L.D.2
Maintenance scheduleProactive at Td/3T_d/3 plus reactive on detection4.3.3, §L.D.3
Register horizon entriesDrift, decay, confidence, wear, contract, rate shortfall3.3.4, applied 4.3.3

Exercises with solutions

E 4.3.3.1. Compute the horizon at which survival falls to 0.90 at p=0.05p = 0.05.

S 4.3.3.1. n=ln⁡0.90/ln⁡0.95=(−0.10536)/(−0.05129)=2.054n = \ln 0.90 / \ln 0.95 = (-0.10536)/(-0.05129) = 2.054. The horizon is two episodes. The lesson: a 5% per-episode rate crosses the 0.90 floor at the second episode, so the interventions are the source of shift survival.

E 4.3.3.2 (transfer). Inter-failure times have a lag-1 autocorrelation of 0.3. What is the diagnosis and the consequence?

S 4.3.3.2. The hazard is time-varying, and the independent law understates the tail. Maintenance must be scheduled against the drift integral, not the average rate. The lesson: autocorrelation is the correlated law’s signature — and, per the lab, it is only detectable when the drift constant is short relative to the mean inter-failure time. An autocorrelation of 0.3 at a 29-episode MTBF implies a drift constant of order a few tens of episodes.

E 4.3.3.3. State the five metrics and their bounds for the running case.

S 4.3.3.3. Shift survival ≥0.90\ge 0.90; MTBF >10> 10 episodes; Tm≤Td/3T_m \le T_d/3; recovery decay >0.80> 0.80; maintenance rate ≤4\le 4 per shift. The lesson: the five are the shift acceptance, and the bounds come from the operator and from the safety factor.

E 4.3.3.4 (diagnostic). A 400-episode suite gives a per-episode success rate of 0.82 with σs=0.02\sigma_s = 0.02, and the extrapolated shift survival is 10−9910^{-99}. Is the point estimate useful?

S 4.3.3.4. No. By (B.2) the log interval at 400 episodes is ±8.3\pm 8.3 decades, so the point estimate is not distinguishable from 10−42.810^{-42.8} or 10−26.210^{-26.2}; at the shift it widens to ±23.9\pm 23.9 decades. The lower bound and the extrapolation contract are the useful statements. The lesson: the interval widens linearly in nn on the log scale, and the point estimate is useless at the horizon.

E 4.3.3.5. Compute the segment survival at R=0.809R = 0.809, p=0.18p = 0.18, and a 20-episode segment. Then compute R20R^{20} and explain the difference.

S 4.3.3.5. q=0.03438q = 0.03438, so (1−q)20=0.4968(1-q)^{20} = 0.4968. And 0.80920=0.014420.809^{20} = 0.01442. The second is the probability that 20 consecutive recoveries all succeed, which is not the probability that a 20-episode segment survives: the segment survives if no episode fails, or if a failing episode is recovered, so the per-episode escalation probability is q=p(1−R)q = p(1-R) and not 1−R1-R. The lesson: the compounding quantity is the escalation rate, and substituting RR for it understates segment survival by a factor of 34 here and by thousands at longer segments.

E 4.3.3.6 (transfer). Four drift sources have time constants 8, 4, 36, and 16 hours. Compute the drift constant and the proactive interval.

S 4.3.3.6. TdT_d is the minimum, 4 hours. The proactive interval is Td/3=1.33T_d/3 = 1.33 hours, which gives 6 intervals and 5 resets per 8-hour shift at a cost of 2.5 hours. The lesson: the smallest constant dominates the schedule, and the schedule cost is proportional to how small it is.

E 4.3.3.7 (diagnostic). The recovery kind distribution shifts toward abort and escalate across shifts. Which degradation mode?

S 4.3.3.7. Recovery state: the retry budget is consumed and the degrade tolerance accumulates. The fix is the reset schedule. The lesson: recovery state is the fourth degradation mode, and its signature is the kind distribution rather than any environmental statistic.

E 4.3.3.8. Write the register entry for the recovery-rate shortfall.

S 4.3.3.8. Failure: composed recovery rate 0.809 against a required 0.9995. Probability: certain, by arithmetic. Consequence: shift survival 3.14×10−183.14 \times 10^{-18} against a 0.90 floor. Mitigation: none by scheduling. Residual: the 0.191 unrecovered fraction. The lesson: a register that omits the constraint the arithmetic violates is a register that has not been read.

E 4.3.3.9 (transfer). The maintenance budget is 1.5 hours and 4 resets at 30 minutes costs 2 hours. What is the fix, and what does it buy?

S 4.3.3.9. Three resets at a 2-hour interval costs 1.5 hours, and shift survival falls from 8.57×10−228.57 \times 10^{-22} to 3.37×10−243.37 \times 10^{-24} — a factor of 254. The lesson: the budget constrains the reset count, the reset count sets the segment length, and the segment length moves survival by orders of magnitude that are still orders short of the floor.


Exit test

You are ready for Module 4.4 when you can, without notes:

  1. Derive the horizon failure law and state its three regimes.
  2. State the drift hazard and explain why it produces clustering.
  3. Compute the recovery rate a stated floor requires, and compare it against 0.809.
  4. Name the five long-horizon metrics and their bounds.
  5. Compute the evaluation window from an interval resolution.
  6. State the three clauses of the extrapolation contract.
  7. Name the four drift sources, their detection channels, and their time constants.
  8. State why the drift constant is the smallest of the four.
  9. State the three maintenance thresholds and their decisions.
  10. Compute a segment survival from pp, RR, and the segment length, and say why RkR^k is wrong.
  11. Name the four degradation modes and their signatures.
  12. State the composed maintenance schedule and its cost.
  13. Assemble the 8-hour shift plan’s numbers.
  14. Read the horizon evidence against its bounds and state the verdict.
  15. Explain why the autocorrelation test has no power at Td=4T_d = 4 hours and what that implies for the claim that clustering is the correlated law’s signature.

Cross-references. 4.3.2 supplies the stopping rule and the residual, restated in §L.A.3 and §L.C.1. 4.3.1 supplies the power arithmetic and the interval lower bound, restated in §L.B.2 and §L.B.3. 4.2.4 supplies the five recovery kinds, the composed rate, and the register, restated in §L.A.3, §L.C.3, and §L.E.3. 4.2.2 supplies the total blind window and the attribution, restated in §L.A.2 and §L.C.2. 3.2.3 supplies the usable horizon and the coverage flag, restated in §L.C.1. 1.1.4 supplies the pipeline health checks, restated in §L.A.2. 3.3.4 supplies the register residuals and the safe state, restated in §L.E.3. Module 4.4 uses the horizon failure law and the maintenance policy as forward-looking defence evidence.


§L.10 — Provenance

Every load-bearing number in this lesson, with its evidence tier and reproduction status. Tier A: measured in the lab on this machine. Tier B: derived from a stated closed form. Tier C: quoted from a prior lesson. Status R: reproduced here; S: stated only; P: partial.

ClaimValueTierStatusSource
Shift length in episodes1,152BR8×3600/258 \times 3600 / 25
Per-episode failure rate0.18CS4.3.2 §L.E.1
Composed recovery rate0.809CS4.2.4 §L.C.3
Escalation rate0.03438BRp(1−R)p(1-R)
Independent survival, raw5.17×10−1005.17 \times 10^{-100}BR(A.1)
Independent survival, recovered3.14×10−183.14 \times 10^{-18}BR(A.1)
Drift survival, no reset1.83×10−361.83 \times 10^{-36}ARlab, surv_drift(1)
Required recovery, zero-failure0.99949BR(A.3)
Required recovery, at most one0.99744BRPoisson inversion
Reset schedule table7 rowsARlab
Lag-1 autocorrelation, Td=4T_d = 4 h+0.010±0.034+0.010 \pm 0.034ARlab, 30 seeds
Lag-1 autocorrelation, Td=0.5T_d = 0.5 h+0.118±0.036+0.118 \pm 0.036ARlab, 30 seeds
Log interval at 400 episodes±8.3\pm 8.3 decadesARlab, checked against Monte Carlo (9.37 vs 9.39 nats)
Window at five-point resolution138.3 shiftsBR(B.1)
Drift constants8, 4, 36, 16 hCS4.3.2 §L.D.1, 3.4.4, 2.3.3, 2.2.4

Corrections to the previous draft of this lesson. Three defects were found by running the lab against the prose, and each is recorded here because the failure mode is general.

  1. The per-segment recovery rate was given as RkR^{k} — 0.80948≈10−50.809^{48} \approx 10^{-5} — with the conclusion that per-segment failure is “essentially certain”. The compounding quantity is the escalation rate q=p(1−R)q = p(1-R), so the segment survival is (1−q)48=0.1865(1-q)^{48} = 0.1865 and the segment failure is 0.81. The substitution of a recovery probability for a survival probability is the error §L.C.3 now warns against explicitly.
  2. The floor arithmetic followed from (1): the draft claimed a 4-reset schedule meets 0.90. It gives 1.08×10−221.08 \times 10^{-22}. The correct conclusion, that no schedule reaches the floor and the recovery rate is the binding constraint, is now the lesson’s central result.
  3. The drift constant was taken as 16 hours (“2 shifts”), described as the smallest of the four, when two of the four constants are 8 and 4 hours. Td=4T_d = 4 hours, and the proactive interval follows as 1.33 hours.

A fourth, smaller defect: the draft’s “5 segments of 230 episodes” does not divide 1,152. The lab now tiles the shift with a remainder segment and asserts the tiling. A fifth: the draft called the 16-hour policy constant the smallest of the four; two of the four are 8 and 4 hours.