Lesson 3.3.2 — Code-as-Policies

Week 24

Prerequisite lessons: 3.3.1 (LLM task planning and decomposition), 3.2.3 (planning with world models), 3.1.4 (VLA fine-tuning), 2.3.2 (motion planning for manipulation), 2.1.2 (object detection and 6D pose estimation), 1.1.4 (the full-stack pick-and-place pipeline).

Prior results used in this lesson. From 3.3.1: the propose–dispose composition, the verifier’s five checks, and the predicate-computation division (the LLM names predicates, the verifier computes them). From 3.2.3: the receding-horizon planner and the four safety checks. From 2.3.2: the clearance-constrained feasible set. From 2.1.2: the 6D pose estimator and its covariance. From 1.1.4: the seven-stage state machine. Each is restated at the point of use.

What this lesson hands forward. The executable-program representation, the primitive library contract, the code-verification layer, and the recovery pattern. Lesson 3.3.3 (grounding language to affordances) treats the primitives’ preconditions as the affordance model. Lesson 3.3.4 (safety and constraints) uses the code verifier as the sandbox. Module 4.2 composes the code layer with the policy layer.

The running case in one sentence. The running arm, gripper, eye-in-hand camera, table, cube, bin, and divider are unchanged; the instruction is the same conditional as 3.3.1, but the output is now a Python program that calls perception and control primitives, and this lesson asks what makes such a program correct, safe, and recoverable.

§L.0 — Lesson map, objectives, and misconceptions

Observable outcomes. By the end of this lesson you can, without notes: (1) state the three properties a program representation adds over a text plan; (2) write the primitive library contract with signatures, preconditions, and postconditions; (3) explain why the primitive set’s granularity determines what the LLM can and cannot compose; (4) write a program for the running case’s conditional instruction; (5) name the four code-generation failure modes and their signatures; (6) design the execution harness with its per-call guards; (7) state the sandboxing rules and the capability-restriction mechanism; (8) write the recovery pattern for a failed primitive; (9) verify a program statically against the primitive library and dynamically against the scene; (10) integrate the code layer into the 1.1.4 state machine as the PROGRAM state.

Misconception inventory.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

Q1. In 3.3.1, the verifier’s five checks were reference, predicate, capability, feasibility, and consistency. Which of them can be checked by static analysis of a program, and which require the scene?

Answer. Reference and capability are static (the primitive library and the scene’s objects are known); predicate, feasibility, and consistency are semantic and require the scene’s geometry and the planner’s feasible set. Used in §L.C.3 to separate the static and dynamic verification passes.

Q2. In 3.2.3, the planner maximizes a predicted return subject to the four safety checks. How does a program’s primitive call relate to that planner?

Answer. A primitive like goto(pose) or grasp(obj) is a wrapper that internally queries the planner; the program composes planner queries, so the planner’s checks apply per call. Used in §L.B.1 to define the primitives’ contracts.

Q3. In 3.3.1, the predicate-computation division was that the LLM names predicates and the verifier computes them. In code-as-policies, what plays the role of “naming the predicate”?

Answer. The program calls a primitive — is_blocked(cube, bin1) — and the primitive’s implementation computes the predicate from the scene; the program names it, the implementation computes it. Used in §L.A.2.

Q4. In 1.1.4, the state machine has health checks at each stage boundary. What does a program’s execution add to the state machine’s observability?

Answer. The program’s call trace — which primitive was called, with what arguments, and what it returned — is a finer-grained log than the state machine’s stage transitions, and it is what the recovery and the audit use. Used in §L.C.2.

Part A — Why code

§L.A.1 — The representation argument

A code-as-policies system (Liang et al. 2023, tier B, S) replaces the LLM’s text plan with an executable program. The program calls primitives — functions that perceive, plan, and actuate — and the composition is Python: loops, conditionals, arithmetic, and function calls.

The representation adds three properties that a text plan does not have.

Executability. A text plan says “pick up the cube”; the program says grasp(get_pose("cube")), and the difference is that the program’s semantics are defined by the primitives. The executor does not interpret the text; it runs the program.

Compositionality with feedback. A program can branch on perception: if is_blocked("cube", "bin1"): target = "bin2" else: target = "bin1". The branch is evaluated at execution time against the scene, not at generation time against the serialization. This is the property that makes the conditional instruction tractable — the 3.3.1 verifier had to evaluate the predicate from the serialization, and the program evaluates it from the live scene.

Arithmetic and iteration. A program can compute a pose from a detected pose — place(get_pose("cube") + offset) — and can loop over objects — for obj in objects_near("table"): .... The text plan cannot express these compositions without a separate language.

The representation’s cost is that the program is only as safe and expressive as the primitive library. A library with a grasp(obj) that can be called on any object, including the table, is a library whose programs can command an unsafe action. The primitive contract of Part B is the response.

Self-explanation 1. Why does the program’s conditional on the live scene matter for the running case’s instruction, when the 3.3.1 verifier already evaluated the condition from the serialization?

Answer. Because the serialization is a snapshot taken before planning, and the scene can change during execution — an operator moves the bin, the cube slips — so the condition must be re-evaluated when the branch is reached, not when the plan was made.

§L.A.2 — The primitive library is the policy

The lesson’s central claim is that the primitive library is where the policy lives. The LLM’s contribution is the composition, and a composition is only as capable as the primitives it composes.

Three consequences follow, and each is a design decision.

Granularity. A primitive that is too coarse — pick_and_place(cube, bin) — removes the LLM’s ability to insert a check or a recovery between the two actions. A primitive that is too fine — set_joint_torque(3, 0.5) — pushes the composition into the LLM’s hands, where it has no model of the dynamics. The right granularity is the sub-task level of 3.3.1’s HTN: goto, grasp, release, is_blocked, get_pose.

Preconditions. Each primitive has a precondition that the executor checks before calling it. grasp(obj) requires is_graspable(obj), which is a geometric predicate computed from the pose and the gripper’s opening. A precondition that is not checked is a safety hole.

Postconditions. Each primitive has a postcondition that the executor checks after it returns. goto(pose) has the postcondition that the end-effector is within tolerance of the pose; a violated postcondition is a primitive failure that the program’s recovery must handle.

The 3.3.1 predicate-computation division is restated here: the program names a precondition, the primitive’s implementation computes it. The program cannot fake the computation, and the implementation cannot fake the call.

Concept check. A team’s library has a primitive push(obj) with no precondition. An LLM program calls push("table"). What happens, and what is the fix?

Answer. The primitive executes a push on the table, which is either a no-op or a dangerous force application depending on the implementation. The fix is a precondition is_pushable(obj) that rejects the table, checked before the call. [displaces M1]

Worked-example ladder.

WE-L.A.a (fully worked). The library has the primitives get_pose(name), goto(pose), grasp(name), release(), is_blocked(a, b), and place_at(pose). Write the program for “put the cube in the bin, but if the divider is in the way, put it in the second bin”.

Solution.

target = "bin2" if is_blocked("cube", "bin1") else "bin1"
goto(grasp_pose("cube"))
grasp("cube")
goto(approach_pose(get_pose(target)))
place_at(get_pose(target))
release()

The program has five calls and one conditional; the conditional is evaluated at execution time.

WE-L.A.b (completion). Add a recovery: if the grasp fails, retry once after re-grounding the pose. Completion.

target = "bin2" if is_blocked("cube", "bin1") else "bin1"
goto(grasp_pose("cube"))
if not grasp("cube"):
    goto(grasp_pose("cube"))
    grasp("cube")
goto(approach_pose(get_pose(target)))
place_at(get_pose(target))
release()

The completion shows that the program’s recovery is a branch, not a re-prompt.

WE-L.A.c (faded). Write the precondition for place_at(pose) and the postcondition for release(). Faded step. place_at requires is_reachable(pose) and is_clear(pose); release has the postcondition not holding(). Completion. The preconditions are geometric predicates computed by the primitive; the postconditions are state checks the executor runs.

WE-L.A.d (contrastive near-miss). A learner’s library has a single primitive do_task(instruction) that internally calls a VLA. The LLM’s programs become single-line calls, and the “code-as-policies” system degenerates to a VLA with an LLM front end. Why it fails: the library’s granularity is too coarse for the LLM to add checks, branches, or recoveries; the composition adds nothing. The fix: granularity at the sub-task level, so the LLM’s composition is where the value is.

Comparative table: representation choices.

Design principle P43.1. The primitive library is the policy; the LLM writes the composition. Design the library’s granularity, preconditions, and postconditions with the same care as the policy itself.

Consolidation checkpoint 1

(a) Three capabilities. You can state the three properties a program adds over a text plan. You can write a program for the running case’s instruction. You can explain the granularity trade.

(b) Transfer probe. A library has 50 primitives, each at the sub-task level. Is that too many for the LLM’s context? Answer: no; the LLM needs the signatures and the docstrings, not the implementations, and 50 signatures fit in a few hundred tokens. The constraint is the primitives’ semantic clarity, not their count.

(c) Remediation rule. If you cannot write the program of WE-L.A.a from the library’s signatures, re-read §L.A.2 before Part B.

Part B — The primitive library contract

§L.B.1 — Signatures, preconditions, postconditions

A primitive’s contract has four parts: its signature (name, arguments, return), its precondition (a predicate over the scene and the arguments that must hold before the call), its effect (what it does), and its postcondition (a predicate that must hold after the call).

The running case’s library has eight primitives, each at the sub-task level:

The table is the contract, and the executor is the enforcer. A call whose precondition is false is rejected before the primitive runs, with a rejection reason that names the failed predicate.

§L.B.2 — The precondition’s two audiences

A precondition serves two audiences, and the distinction matters for the design.

The LLM’s audience. The LLM reads the precondition as a constraint on which programs are legal. A precondition that is false in every scene — is_graspable("table") — tells the LLM not to write that call. The precondition’s documentation is what shapes the LLM’s composition.

The executor’s audience. The executor reads the precondition as a runtime check. A call whose precondition fails at execution is rejected regardless of whether the LLM “knew” it would fail. The precondition’s implementation is what keeps the robot safe.

A precondition that is documented but not implemented is a lie; a precondition that is implemented but not documented is a surprise. The contract requires both, and the running case’s library has a test that every documented precondition is enforced.

§L.B.3 — The effect and the postcondition

The effect is what the primitive does; the postcondition is what must be true after. The postcondition is the executor’s mechanism for detecting a primitive failure that the primitive did not report.

Consider goto(pose). The primitive might time out, hit an obstacle, or be interrupted by a safety check. In each case, the primitive’s return value could be True (it did not notice the failure) or False (it did). The postcondition check — dist(ee, pose) < tol — is independent of the primitive’s self-report, and it catches the case where the primitive’s self-report is wrong.

The postcondition’s tolerance is the design parameter. Too tight and the check rejects every call because of control noise; too loose and the check passes a failed call. The running case’s goto uses a 5 mm tolerance, calibrated from the 2.3.3 controller’s steady-state accuracy.

Self-explanation 2. Why is the postcondition check necessary if the primitive already returns a boolean?

Answer. Because the primitive’s boolean is its own assessment, which can be wrong; the postcondition is an independent measurement of the effect, and the independence is what makes the check meaningful.

Concept check. place_at(pose) returns True and the cube is still in the gripper. Which check catches this, and what does it do?

Answer. The postcondition at_place_pose(pose) combined with the holding() state check; the executor sees that the gripper is still closed and treats the primitive as failed, then the program’s recovery runs. [displaces M2]

Worked-example ladder.

WE-L.B.a (fully worked). grasp("cube") is called when the end-effector is 3 cm from the grasp pose, and the precondition is dist(ee, grasp_pose) < 1 cm. Compute the rejection and its reason.

Solution. The precondition is false; the executor rejects with reason precondition failed: dist(ee, grasp_pose)=0.03 > 0.01. The program’s recovery must move the end-effector closer before retrying.

WE-L.B.b (completion). The program’s recovery calls goto(grasp_pose("cube")) and then grasp("cube"). Write the postcondition check that confirms the second call succeeded. Completion. After grasp, check holding("cube"), which is a state predicate the executor evaluates from the gripper’s force reading and the 2.1.2 pose tracking.

WE-L.B.c (faded). Write the precondition for release() and the two cases in which it can fail. Faded step. Precondition holding(); failures are not holding (no-op or error) and the target pose not reached (release would drop the object). Completion. The second case is why release should be preceded by a postcondition check on place_at, not called directly.

WE-L.B.d (contrastive near-miss). A learner’s goto has a postcondition with a 1 µm tolerance and rejects every call because the controller’s noise is 2 mm. Why it fails: the tolerance was set from the controller’s specification rather than from its steady-state behaviour, and the specification is an ideal that the noise violates. The fix: calibrate the tolerance from measured steady-state error, or make the tolerance a percentage of the pose’s precision requirement.

Comparative table: the four contract parts.

Design principle P43.2. A precondition that is not enforced is a lie, and a postcondition that is not independent of the primitive’s self-report is not a check.

Consolidation checkpoint 2

(a) Three capabilities. You can write a primitive’s four-part contract. You can explain the precondition’s two audiences. You can set a postcondition tolerance from measured behaviour.

(b) Transfer probe. The primitive place_at(pose) has no postcondition. A program that calls it silently fails when the pose is unreachable. Which check catches it? Answer: none, unless the program checks holding() afterwards; the fix is to add the postcondition, which makes the primitive fail explicitly.

(c) Remediation rule. If you cannot write a precondition and a postcondition for a primitive of your choice, re-read §L.B.1 before Part C.

Part C — Generation, execution, and recovery

§L.C.1 — The generation prompt

The LLM generates a program from four inputs: the instruction, the scene serialization (restated from 3.3.1 §L.C.3), the primitive library’s signatures and docstrings, and a small set of worked examples.

The examples are the few-shot prompt. A good example shows: a conditional branch, a loop over objects, an error check, and a recovery. The examples are the LLM’s evidence of what a program looks like, and their coverage of the primitives is the coverage of what the LLM will generate.

The library’s docstrings are the LLM’s specification. A docstring that says “grasp the named object” without the precondition is a docstring that invites a call on an un-graspable object. The docstring should restate the precondition in the LLM’s language, and the running case’s library’s docstrings do.

The scene serialization is the same as 3.3.1’s, with one addition: the objects’ names are the strings the program will use. A mismatch between the serialization’s names and the program’s strings is a runtime reference error, and the static checker of §L.C.3 catches it.

§L.C.2 — The execution harness

The executor runs the program step by step, and each step has four guards.

Static guard. Before any execution, the program is parsed and every call’s signature and reference is checked. An unknown primitive or an unknown object name is a rejection.

Precondition guard. Before each call, the precondition is evaluated against the live scene. A false precondition is a rejection with the predicate and its value.

Call guard. The call is executed with a timeout and a force limit. A timeout or a force-limit violation is a primitive failure.

Postcondition guard. After each call, the postcondition is evaluated. A false postcondition is a primitive failure, even if the primitive returned True.

The guards are the executor’s mechanism, and their logs are the program’s call trace: for each call, the primitive, the arguments, the guard outcomes, and the return. The trace is what the recovery and the audit use.

Recovery is the program’s own branch on a failed call. The program can retry (after a state change), re-ground (re-fetch the pose), or switch branches. A retry without a state change is a loop, and the executor’s retry counter — a maximum of two per call — stops it.

§L.C.3 — The static and dynamic verification

The static verification is what can be checked without the scene: signatures, references, and the primitive library’s capabilities. It is the 3.3.1 verifier’s reference and capability checks, applied to a program.

The dynamic verification is what requires the scene: preconditions, postconditions, and the geometric predicates. It is the 3.3.1 verifier’s predicate, feasibility, and consistency checks, applied to a program.

The two passes are separate because they run at different times. The static pass runs on the generated program, before execution; it catches the LLM’s signature errors and hallucinated APIs. The dynamic pass runs at each call, during execution; it catches the geometric and state errors that the serialization could not resolve.

The static pass’s coverage is what makes code-as-policies safer than text planning: a text plan’s references are checked by the 3.3.1 verifier, but its calls are not, because a text plan has no calls. A program’s calls are checked before any execution.

Self-explanation 3. Why does the static pass check the references and not only the signatures?

Answer. Because a program can call grasp("cube") when the scene has no cube named “cube” — the signature is valid and the reference is not; the static pass checks both against the scene’s names.

Concept check. A program calls goto(get_pose("cube")) and the get_pose call’s postcondition is cov < 0.01. The pose is returned with cov = 0.03. Which guard fires, and what does the program do?

Answer. The postcondition guard fires on get_pose, and the program’s recovery re-observes the cube, which the 2.1.5 visual odometry can do by moving the camera; if the covariance remains high, the program reports an uncertainty failure. [displaces M3]

Worked-example ladder.

WE-L.C.a (fully worked). A program calls grasp("cube") twice without changing state, and both calls return False. Compute the retry count and the executor’s action.

Solution. The retry counter reaches 2, which is the maximum; the executor rejects the third call and returns a RetryExhausted failure with the call trace. The program’s recovery must change the state, not retry.

WE-L.C.b (completion). Write the recovery that changes the state: re-ground the cube’s pose and move closer, then retry. Completion.

if not grasp("cube"):
    goto(grasp_pose("cube"))     # re-ground via get_pose
    grasp("cube")                # retry with a state change

The completion shows that the recovery is a program, not a re-prompt.

WE-L.C.c (faded). Write the static checker’s rules for a reference error and a signature error, and give an example of each. Faded step. Reference: grasp("cube") when "cube" not in scene_names. Signature: goto("bin1") when goto takes a pose. Completion. Both are caught before execution, and both are the LLM’s errors, not the scene’s.

WE-L.C.d (contrastive near-miss). A learner’s executor runs the program with no postcondition checks, and a goto that times out is treated as success because the primitive returned True on timeout. The program proceeds to grasp from 10 cm away. Why it fails: the primitive’s self-report was wrong, and the postcondition was absent. The fix: add the postcondition, and treat timeout as failure regardless of the return.

Comparative table: the executor’s four guards.

Design principle P43.3. Verify statically before execution and dynamically at every call; the two passes catch different errors, and neither alone is sufficient.

Consolidation checkpoint 3

(a) Three capabilities. You can write the generation prompt’s four inputs. You can write the four guards and their order. You can write a recovery that changes state.

(b) Transfer probe. A program’s place_at returns True and the cube is on the table, not in the bin. Which guard catches it? Answer: the postcondition, if it checks the cube’s pose against the bin’s; the primitive’s self-report is not enough, and the postcondition must be an independent measurement.

(c) Remediation rule. If you cannot state why the static pass checks references as well as signatures, re-read §L.C.3 before Part D.

Part D — Verification, sandboxing, and safety

§L.D.1 — The sandbox’s scope

The generated program is untrusted code. It comes from an LLM whose training objective is plausibility, and a plausible program can do anything the interpreter can do. The sandbox’s purpose is to restrict what the program can do to what the primitive library permits.

The sandbox’s rules are:

No imports. The program cannot import modules; the primitives are the only functions in scope.

No filesystem or network access. The program cannot read or write files, or open connections; a program that tries is rejected at parse time.

No reflection. The program cannot call eval, exec, getattr, or any mechanism that would let it construct a call the static checker did not see.

Bounded loops. The program’s loops have a maximum iteration count, set by the executor, so a program cannot run forever.

Bounded recursion. The program’s recursion depth has a maximum, for the same reason.

Timeout. The whole program has a wall-clock timeout, and exceeding it is a rejection.

The sandbox’s scope is what the program can express, not what the primitives do. A program that passes the sandbox can still call grasp("table") if the primitive’s precondition permits it. That is why the preconditions of Part B are the semantic guard, and the sandbox is the syntactic one.

§L.D.2 — The safety layer

The safety layer composes three mechanisms from the previous lessons.

The primitive preconditions are the semantic guard: a call whose precondition is false is rejected before the primitive runs. The 2.3.3 force limits are the precondition’s implementation for grasp and place_at.

The 3.2.3 safety checks are the planner’s guard: the primitive’s internal planner query is subject to the four checks, so a program whose goto would cross the divider is rejected by the planner, not by the program.

The 2.2.4 Q-shield is the runtime guard: any action whose predicted value falls below the threshold is replaced by the safe fallback, regardless of which primitive issued it.

The three mechanisms are redundant by design. A program that bypasses the precondition is caught by the planner; one that bypasses the planner is caught by the shield; one that bypasses the shield is a hardware fault, which is why the hardware’s physical limits are the last line.

Self-explanation 1. Why is the sandbox insufficient without the preconditions?

Answer. Because the sandbox restricts the program’s language, not the semantics of its calls; a program can be syntactically safe and semantically unsafe, calling grasp on an object that should not be grasped.

Concept check. A program passes the sandbox and calls goto(pose) with a pose inside the divider. Which mechanism rejects it, and what is the rejection reason?

Answer. The 3.2.3 planner’s collision check, because the pose violates the 2.3.2 clearance; the rejection reason is the clearance violation. [displaces M2]

§L.D.3 — The four code-generation failure modes

The LLM’s program can fail in four ways, and each has a signature in the call trace.

Signature error. The program calls a primitive with the wrong argument types or count. The static checker catches it before execution. Cause: the LLM’s model of the library’s signatures. Fix: more examples, or a type-aware prompt.

Reference error. The program calls a primitive with an object name not in the scene. The static checker catches it. Cause: hallucination or a stale serialization. Fix: re-prompt with the scene’s names.

Logic error. The program’s composition is wrong — the conditional’s branches are swapped, the loop’s order is inverted, the recovery is missing. The error is visible only in execution, as a wrong outcome with a valid trace. Cause: the LLM’s reasoning over the scene. Fix: re-prompt with the failure and the trace, or verify the program’s logic against a spec.

Semantic error. The program’s calls are valid and the outcome is wrong because a precondition was true at the time of the check and false at the time of the call — a race with the environment. The postcondition catches it. Cause: the scene changed between the check and the call. Fix: re-check the precondition immediately before the call, or hold the scene with a lock.

The four modes are ordered by how early they are caught: signature and reference at parse, logic at execution’s outcome, semantic at execution’s postcondition. The later the catch, the more expensive the recovery.

Self-explanation 2. Why is the semantic error the hardest to catch, even with postconditions?

Answer. Because it is a race: the precondition and the call are separated in time, and the environment can change in between; the postcondition catches the outcome, but only after the primitive has run.

Worked-example ladder.

WE-L.D.a (fully worked). A program calls goto(get_pose("cube")) and the get_pose call returns a pose with a high covariance. The goto primitive’s precondition is is_clear(pose), which the covariance does not affect. The arm moves to a pose that is 4 cm off the cube. Diagnose the failure.

Solution. The failure is a logic error: the program used a pose whose uncertainty was too high for the grasp tolerance. The precondition is_clear is about obstacles, not uncertainty; the missing check is on the pose’s covariance. The fix is to add a precondition cov < 0.01 to goto when the target is a grasp pose, or to make the program check the covariance before calling.

WE-L.D.b (completion). Add the covariance check to the program and write the recovery. Completion.

p = get_pose("cube")
if p.cov > 0.01:
    look_closer("cube")      # an observation primitive
    p = get_pose("cube")
goto(grasp_pose(p))

The completion shows that the program’s recovery is a state change, not a retry.

WE-L.D.c (faded). Write the sandbox’s parse-time rules and give an example of a program rejected by each. Faded step. Import rule: import os rejected. Reflection rule: eval("grasp('cube')") rejected. Loop bound: while True: goto(...) rejected. Completion. The three rules cover different attack surfaces, and each is checked at a different parse stage.

WE-L.D.d (contrastive near-miss). A learner’s program calls is_blocked("cube", "bin1") and then, 200 ms later, calls goto with a path that was clear at the check and blocked by the time of the call because the operator moved the bin. Why it fails: the semantic error — the check and the call are separated in time. The fix: re-check the predicate immediately before the call, or have the goto primitive’s internal planner re-check the path at execution.

Comparative table: failure modes and their catches.

Design principle P43.4. The sandbox is syntactic, the preconditions are semantic, and the planner’s checks are the physical line; a program is safe only when all three hold.

Consolidation checkpoint 4

(a) Three capabilities. You can list the sandbox’s rules. You can order the three safety mechanisms. You can diagnose the four failure modes from their signatures.

(b) Transfer probe. A program passes the sandbox and the preconditions, and the arm collides with the divider. Which mechanism failed? Answer: the 3.2.3 planner’s collision check, because the goto primitive’s planner query was supposed to catch it; if it did not, the primitive’s planner integration is the bug.

(c) Remediation rule. If you cannot state why the sandbox is insufficient without the preconditions, re-read §L.D.1 before Part E.

Part E — The running case

§L.E.1 — The program

The instruction is the same conditional as 3.3.1, and the program is the one of §L.A.2, extended with the covariance check and the recovery.

p = get_pose("cube")
if p.cov > 0.01:
    look_closer("cube")
    p = get_pose("cube")
target = "bin2" if is_blocked("cube", "bin1") else "bin1"
goto(grasp_pose(p))
if not grasp("cube"):
    goto(grasp_pose(get_pose("cube")))
    grasp("cube")
goto(approach_pose(get_pose(target)))
place_at(get_pose(target))
if not release():
    place_at(get_pose(target))
    release()

The program has one conditional for the target, one for the covariance, and two recoveries. Every call’s precondition is enforced by the executor, and every call’s postcondition is checked.

§L.E.2 — The execution trace

The executor’s trace for a successful run has eleven entries: get_pose (cov check passes), is_blocked (returns true), goto (precondition passes, postcondition passes), grasp (returns true, postcondition holding passes), goto (approach pose), place_at, release. The trace is the audit record; a failed run’s trace shows the exact call and the guard that failed.

The trace’s value is diagnostic. A run that fails at grasp’s postcondition is a grasping problem; one that fails at goto’s precondition is a planning problem; one that fails at is_blocked’s evaluation is a scene-serialization problem. The trace localizes the failure to a primitive and a guard.

§L.E.3 — The acceptance criteria

A program is accepted for deployment if it satisfies four criteria, and they are the lesson’s operational answer.

Correctness. The program achieves the instruction on the nominal scene, verified by the outcome check.

Robustness. The program achieves the instruction on at least 80% of the scene variations — the cube’s position, the bin’s position, the divider’s presence — and its failures are the recoveries’ failures, not the program’s.

Safety. No call’s precondition is violated in any run, including the failed ones; a precondition violation is a deployment blocker regardless of the success rate.

Traceability. Every run’s trace is logged, and a failed run’s trace names the primitive and the guard that failed.

The four criteria are the code-as-policies analogue of 3.1.4’s conjunction. Correctness alone is insufficient; a program that succeeds on the nominal scene and violates a precondition on a variation is not deployable.

Self-explanation 1. Why is a precondition violation a deployment blocker even when the run succeeds?

Answer. Because the success was luck: the precondition’s violation means the primitive ran outside its contract, and the outcome depended on the violation not mattering, which is not a property the deployment can rely on.

Worked-example ladder.

WE-L.E.a (fully worked). The program succeeds on 92% of 100 scene variations, and 2 of the 8 failures are precondition violations (the grasp was called when the gripper was not at the grasp pose). Compute the acceptance decision.

Solution. The correctness criterion is met (92% > 80%), but the safety criterion is violated (2 precondition violations). The program is rejected; the fix is to make the grasp precondition check the pose tolerance more strictly, or to have the program’s goto verify the pose before the grasp call.

WE-L.E.b (completion). After the fix, the program succeeds on 88% with 0 precondition violations. Recompute the decision. Completion. Correctness 88% > 80%, safety 0 violations, robustness meets the threshold, traceability is in place; the program is accepted.

WE-L.E.c (faded). The program’s is_blocked call is evaluated on a stale serialization and returns False when the divider has moved. Write the change that catches the staleness. Faded step. Add a timestamp to the serialization and reject a program whose calls are more than 100 ms older than the scene. Completion. The is_blocked primitive is re-evaluated against the live scene at the call, which is the program’s advantage over the text plan; the staleness is caught by the primitive’s implementation, not by the program.

WE-L.E.d (contrastive near-miss). A learner accepts the program on the nominal scene and deploys it. The first variation — the cube 10 cm to the left — causes a precondition violation at grasp because the program’s goto was to a pose computed from a stale get_pose. Why it fails: the correctness criterion was measured only on the nominal scene, and the robustness criterion was absent. The fix: test on the variations, and add the covariance check to the program.

Comparative table: the running case’s program at a glance.

Design principle P43.5. Accept a program on correctness, robustness, safety, and traceability together; a success that violates a precondition is a deployment blocker, not a success.

Consolidation checkpoint 5

(a) Three capabilities. You can write the running case’s program with its recoveries. You can read a trace and localize a failure. You can apply the four acceptance criteria.

(b) Transfer probe. The program succeeds on the nominal scene and fails on every variation. Which criterion fails, and what is the first fix? Answer: robustness; the first fix is to test on the variations and add the covariance and pose checks that the nominal scene did not require.

(c) Remediation rule. If you cannot state why a precondition violation is a blocker when the run succeeded, re-read §L.E.3 before the lab.

Failure taxonomy

The table covers the three lab corruptions plus three failures not exercised in the lab: loops, timeouts, and unsafe success.

The lab

Task. Implement the execution harness with the four guards, run the running case’s program on a stub scene with a fault injector, and measure the failure-mode detection rate and the recovery success rate.

Conditions. Python 3.10+, NumPy. The scene is a dictionary; the primitives are stubs that can be made to fail on command. Seed all RNGs.

Controls. (i) The nominal harness with all four guards; (ii) the harness with the postcondition guard disabled; (iii) the harness with the precondition guard disabled; (iv) the harness with the sandbox disabled.

Metrics.

Pre-registered hypotheses.

H1. Control (i) detects at least 90% of the injected faults, because the four guards cover signature, reference, precondition, and postcondition failures.

H2. Control (ii) misses the silent primitive failures — the ones where the primitive returns True and the effect did not happen — and its unsafe-call count is not the failure.

H3. Control (iii) executes calls that violate preconditions, and its unsafe-call count rises above zero.

H4. Control (iv) is unsafe in a way the other controls are not: a program that imports a module can execute arbitrary code.

Break-it sequence.

1. Make a primitive lie silently. Return True from a grasp that did not grasp. Silent: the primitive’s self-report is the only signal unless the postcondition checks.
2. Remove an object from the scene silently. A program that references it is caught by the static checker. Silent: the LLM’s program looks valid.
3. Stale the scene silently. Change the scene’s geometry between the precondition check and the call. Silent: the precondition was true at the check and false at the call.

Acceptance criteria.

What good looks like. A good artefact has five properties. (1) The four guards are separate functions with separate logs, so the detection rate is reported per guard. (2) The recovery is a program that changes the state, and the retry bound is visible. (3) The break-it results are a table. (4) The sandbox is a parse-time AST check, not a runtime filter. (5) The acceptance criteria are the four of §L.E.3, reported together.

A runnable core, seeded and annotated:

# lab_3_3_2.py — code-as-policies harness with four guards
import numpy as np, ast, time
np.random.seed(0)

SCENE = {"cube": {"pose": (0.5, 0.2, 0.3), "cov": 0.005},
         "bin1": {"pose": (0.4, -0.3, 0.15)},
         "bin2": {"pose": (0.2, -0.4, 0.15)}}
FAULTS = {"grasp_lies": False, "stale_scene": False}

def static_check(src, scene_names):
    tree = ast.parse(src)                              # parse
    allowed = {"get_pose", "goto", "grasp", "release",
               "place_at", "is_blocked", "approach_pose", "grasp_pose"}
    for node in ast.walk(tree):
        if isinstance(node, (ast.Import, ast.ImportFrom)):      # no imports
            raise ValueError("sandbox: imports not allowed")
        if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
            if node.func.id not in allowed:                     # known primitives
                raise ValueError(f"static: unknown primitive {node.func.id}")
    for s in ast.walk(tree):                                    # object names
        if isinstance(s, ast.Constant) and isinstance(s.value, str):
            if s.value not in scene_names and s.value not in ("bin1", "bin2"):
                raise ValueError(f"static: unknown reference {s.value}")

class Harness:
    def __init__(self, scene):
        self.scene = scene; self.trace = []; self.retries = {}
    def check_pre(self, name, *args):
        if name == "grasp":
            obj = args[0]; return obj in self.scene and self.at_grasp_pose(obj)
        return True
    def check_post(self, name, ret, *args):
        if name == "grasp" and not FAULTS["grasp_lies"]:
            return ret and self.holding == args[0]
        if name == "grasp" and FAULTS["grasp_lies"]:
            return self.holding == args[0]                     # independent check
        return True
    def call(self, name, *args, **kw):
        self.retries[name] = self.retries.get(name, 0) + 1
        if self.retries[name] > 2:
            raise RuntimeError(f"retry exhausted: {name}")
        if not self.check_pre(name, *args):
            self.trace.append((name, args, "pre-fail")); return False
        ret = self.primitives[name](*args, **kw)
        self.trace.append((name, args, ret))
        if not self.check_post(name, ret, *args):
            self.trace.append((name, args, "post-fail")); return False
        return ret
    # ... primitives stubbed for the lab

harness = Harness(SCENE)
src = '''
p = get_pose("cube")
target = "bin2" if is_blocked("cube", "bin1") else "bin1"
goto(grasp_pose(p))
grasp("cube")
goto(approach_pose(get_pose(target)))
place_at(get_pose(target))
release()
'''
try:
    static_check(src, set(SCENE) | {"bin1", "bin2"})
    print("static check passed")
except ValueError as e:
    print(e)

Lines 8–20 are the static checker: parse, reject imports, reject unknown primitives, reject unknown references. Lines 22–40 are the harness: precondition check before the call, postcondition check after, retry counter with a bound of two. Line 42 constructs the harness, and lines 44–55 are a program that the static checker validates before execution.

Cheat sheet

Exercises with solutions

E 3.3.2.1. Write the three properties a program representation adds over a text plan, and one example of each.

S 3.3.2.1. Executability (grasp("cube") has a defined effect); compositionality with live feedback (if is_blocked(...) at execution time); arithmetic and iteration (place_at(get_pose("cube") + offset)). The lesson: the program’s value is in the composition, not in the text.

E 3.3.2.2 (transfer). A library’s grasp has no precondition. Write the precondition that prevents a call on the table and state its implementation.

S 3.3.2.2. Precondition is_graspable(obj) and at_grasp_pose(obj); the implementation is a geometric check on the object’s size, the gripper’s opening, and the distance to the grasp pose. The lesson: the precondition is a geometric predicate, not a name check.

E 3.3.2.3. Write the postcondition for goto and state why it must be independent of the primitive’s return.

S 3.3.2.3. dist(ee, pose) < tol. It must be independent because the primitive’s return is its own assessment, which can be wrong on timeout or interruption. The lesson: the check is a measurement, not a report.

E 3.3.2.4 (diagnostic). A program calls grasp("cube") when the gripper is 5 cm from the grasp pose and the precondition is dist < 1 cm. Which guard fires, and what is the recovery?

S 3.3.2.4. The precondition guard; the recovery is goto(grasp_pose("cube")) and then grasp. The lesson: the recovery is a state change, not a retry.

E 3.3.2.5. List the sandbox’s rules and give one program rejected by each.

S 3.3.2.5. No imports (import os), no reflection (eval(...)), bounded loops (while True: ...), bounded recursion, timeout. The lesson: each rule closes a different attack surface, and they are checked at parse time.

E 3.3.2.6 (transfer). A program passes the sandbox and the preconditions, and the arm collides with the divider. Which mechanism is the bug in, and why?

S 3.3.2.6. The 3.2.3 planner’s collision check, because the goto primitive’s planner query should have rejected the pose; the bug is in the primitive’s planner integration, not in the program. The lesson: the sandbox and preconditions are not substitutes for the planner’s checks.

E 3.3.2.7. Diagnose a semantic error from the trace and propose the fix.

S 3.3.2.7. The trace shows is_blocked returning False and the goto colliding; the predicate was true at the check and false at the call, or the predicate’s evaluation was stale. The fix is to re-evaluate the predicate immediately before the call, or to have the primitive’s planner re-check at execution. The lesson: the semantic error is a race, and the fix is to shrink the window.

E 3.3.2.8 (diagnostic). A program succeeds on the nominal scene and violates a precondition on a variation. Is it deployable? Justify.

S 3.3.2.8. No: the safety criterion is violated, and the success was luck. The fix is to make the precondition stricter or to add the check the variation required. The lesson: success is not deployment; the four criteria are conjunctive.

E 3.3.2.9. Compute the detection rate of the four guards for the failure modes signature, reference, logic, semantic.

S 3.3.2.9. Static catches signature and reference; precondition catches semantic if the race window is zero; postcondition catches logic if the outcome is checked and semantic if the effect is checked. No single guard catches all four; the composition’s detection rate is the union. The lesson: the four guards are complementary, not redundant.

Exit test

You are ready for Lesson 3.3.3 when you can, without notes:

1. State the three properties a program adds over a text plan.
2. Write a primitive’s four-part contract.
3. Explain the precondition’s two audiences.
4. State why the postcondition must be independent.
5. Write the four guards in order and state what each catches.
6. Write a recovery that changes the state.
7. State the retry bound and why it exists.
8. List the sandbox’s rules and the attack each closes.
9. Order the three safety mechanisms and explain each’s role.
10. Diagnose the four code-generation failure modes from their traces.
11. Explain the semantic race and its fix.
12. Write the running case’s program with its recoveries.
13. Read a trace and localize a failure.
14. Apply the four acceptance criteria as a conjunction.
15. Integrate the code layer as the PROGRAM state in the 1.1.4 pipeline.

Cross-references. 3.3.1 supplies the propose–dispose composition and the predicate-computation division restated in §L.A.2 and §L.B.1. 3.2.3 supplies the planner and the four safety checks restated in §L.D.2. 2.3.2 supplies the clearance constraint restated in §L.D.2. 2.3.3 supplies the force limits restated in §L.D.2. 2.1.2 supplies the pose estimator and its covariance restated in §L.B.1 and §L.E.1. 1.1.4 supplies the state machine extended in §L.E.3. Lesson 3.3.3 treats the primitives’ preconditions as the affordance model. Lesson 3.3.4 uses the sandbox and the verifier as the safety layer for LLM outputs.

Ready. The next lesson to write is 3.3.3 — Grounding Language to Affordances. Say “next” to generate it.