Weeks 1–2. Prerequisite: Part 0 (the Mathematical Prelude).

1.0 Module map and the rationale for build-first

This module inverts the usual sequence. You will build a working minimal harness before you are given its formal theory, then formalise what you have built. The reason is pedagogical, not stylistic. The formal objects of agent theory — policies, beliefs, observation functions, transition kernels — are abstract until you have watched a loop stall because a tool returned an error the harness did not parse, or watched a context window silently drop the one observation that determined the answer. Building first gives those abstractions a referent.

The module has two parts and two labs.

By the end of the module you will be able to do five things that the rest of the curriculum assumes:

1. Specify a harness precisely enough that another engineer could implement it without asking you questions.
2. Instantiate the POMDP tuple $(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{O},\mathcal{Z},R,\gamma)$ for a concrete harness, naming every component.
3. Prove that a given context-management policy is lossy, and identify what it loses.
4. Classify a task as workflow, single agent, or multi-agent using a decision procedure grounded in task properties rather than fashion.
5. Explain why the harness, not only the model, determines performance — formally, not rhetorically.

Everything in this module is self-contained. Where a result from Part 0 is used, it is restated in the form needed here.

PART A — THE HARNESS

1.1 What a harness is: the four necessary elements

1.1.1 The working formula

The discipline’s working formula is:

The model is the learned function that maps a context to a next action (usually a tool call or a text continuation). The harness is everything else: the loop that calls the model, the tools it can call, the way context is assembled and trimmed, the rules that govern termination, permission, and error recovery, and the instrumentation that records what happened.

This formula is not a slogan. Section 1.4 proves that changing the harness changes the POMDP the model is solving, and therefore changes expected return even when the model is held fixed. That is the formal content of “the harness is load-bearing.”

1.1.2 Definition of a harness

Definition 1.1.1 (Harness). A harness is a runtime that mediates between a model and an environment. It consists of four necessary elements:

1. An agent loop — the iteration that calls the model, dispatches its actions, and feeds observations back.
2. A tool interface — the typed action space exposed to the model, and the dispatcher that executes actions and serialises results.
3. Context management — the function that assembles, orders, trims, and formats the history into the context presented to the model at each step.
4. Control mechanisms — termination conditions, permission enforcement, error handling, retry and recovery policy, and budget limits.

This definition is the one formalised in the source-code study of eleven production harnesses (arXiv:2609.00006, evidence tier S: single-lab, non-peer-reviewed, but methodology inspectable and the scaffold released). The study reports that all eleven systems contain these four elements, and that their differences lie in how the elements are implemented, not in whether they are present.

1.1.3 Why these four are necessary

The claim of necessity is an argument, not an empirical observation. Remove any element and a property required for agency is lost.

Remove the agent loop. The model produces one action and stops. The system can respond but cannot act over time. It is a classifier or a generator, not an agent. Property lost: iteration.

Remove the tool interface. The model produces text but nothing in the environment changes. The system can describe but cannot act. Property lost: effect on the environment. (A pure-chat system is a degenerate case: the “tool” is the message channel to the user. The element is present, minimally.)

Remove context management. The context grows without bound. Two consequences follow. First, the context eventually exceeds the model’s window and the harness must either truncate arbitrarily or fail — both are forms of context management, so the element cannot actually be removed, only done badly. Second, even within the window, the model’s attention degrades with length and with distractor content (the Lost in the Middle result, arXiv:2307.03172, tier R). Property lost: a usable, bounded observation function. Context management is unavoidable; the only question is whether it is designed or accidental.

Remove control mechanisms. The loop has no termination condition, so it runs until the budget is exhausted or the process is killed. It has no permission enforcement, so any tool call the model emits is executed. It has no error handling, so a single malformed tool response crashes the run. Property lost: safe termination and safe action.

The four elements are therefore necessary in the precise sense that their absence destroys a property that distinguishes an agent from a non-agent. This is why the definition is constitutive rather than stipulative: it carves the space of runtimes at the joint where agency lives.

1.1.4 The ETCLOVG preview

A separate survey literature proposes a seven-layer taxonomy for analysing harnesses: Execution, Tooling, Context, Lifecycle, Observability, Verification, Governance. You do not need it yet. It is named here so that you recognise it in Module 6, where it is the organising device. Note the relationship: the four necessary elements are the constitutive core (what makes it a harness); the seven ETCLOVG layers are an analytical decomposition (how to study one). The first answers “what is it?”; the second answers “what are its parts, in more detail?”

1.1.5 Concept check

Why is “context management” necessary rather than optional, given that a model has a finite context window?

Answer. Because the window is finite, some function must decide what enters it. If the harness does not decide deliberately — by truncation policy, summarisation, retrieval, or ordering — then the decision is made accidentally by the order in which events happen to arrive, which is the worst possible policy. The element cannot be removed; it can only be designed or neglected. Neglected context management is still context management, and it is the kind that fails silently.

1.2 The minimal-viable-harness specification

This section gives a complete operational specification of a minimal harness satisfying Definition 1.1.1. It is language-agnostic. Lab 1a asks you to implement it in your production language of choice.

The scaffold is deliberately small — the reference version in the eleven-system study is described as roughly ninety lines. The point is not to build a product; the point is to build something small enough that you can hold all four elements in your head simultaneously, so that when Part B formalises them you recognise every piece.

1.2.1 Data structures

Define five record types. Their fields are normative: the acceptance criteria in §1.2.6 test for them.

Message. A single item in the conversation history.

• role: one of system, user, assistant, tool.
• content: a string (the text).
• tool_call: optional; present only on assistant messages that request a tool.
• tool_result: optional; present only on tool messages.
• timestamp: monotonic counter, used for trace ordering.

ToolCall. A structured action.

• id: a harness-assigned unique identifier.
• name: the tool name; must be a member of the registered tool set.
• arguments: a structured object; must validate against the tool’s argument schema.

ToolResult. The outcome of executing a ToolCall.

• call_id: the identifier of the originating call.
• status: one of ok, error, timeout, denied.
• content: a string (the serialised result, or the error message).
• duration: wall-clock time.

Context. The object handed to the model at each step.

• messages: an ordered list of Message records, already formatted for the model.
• token_estimate: an integer estimate of the context length.
• dropped: a list of records describing what context management removed and why.

State. The harness’s own bookkeeping, not shown to the model.

• step: integer step counter.
• budget_remaining: tokens, tool calls, wall-clock seconds, or dollars — whichever limits are configured.
• consecutive_errors: integer, reset on success.
• terminated: boolean, plus a termination_reason.

1.2.2 The agent loop

The loop is the first necessary element. Its operational semantics are fixed; the implementation language is not.

LOOP:
  1. ASSEMBLE context from history via the context manager.
  2. CHECK control preconditions (budget, step limit, loop detection).
     If any is violated: terminate with the corresponding reason.
  3. CALL the model with the context.
  4. PARSE the model output into either a final answer or one or more ToolCalls.
     If parsing fails: record a parse error, append a corrective message,
     increment consecutive_errors, and CONTINUE.
  5. FOR EACH ToolCall:
       a. VALIDATE the call against the tool schema.
          If invalid: produce a ToolResult with status = error and CONTINUE.
       b. CHECK permission policy.
          If denied: produce a ToolResult with status = denied and CONTINUE.
       c. EXECUTE the tool with a timeout.
          On timeout: status = timeout.
          On exception: status = error with the exception message.
       d. APPEND the ToolResult to history.
  6. IF the model produced a final answer: terminate with reason = completed.
  7. OTHERWISE: increment step and GOTO 1.

Three properties of this loop deserve emphasis because they are where minimal harnesses usually go wrong.

Property 1: every failure becomes an observation. A parse failure, a validation failure, a permission denial, a timeout, and an exception all produce a record appended to history. The model sees them. This is what makes error recovery possible: the model can read “your call to read_file was denied because the path is outside the workspace” and try a different path. A harness that crashes on error, or that logs the error out-of-band without showing it to the model, has removed the model’s ability to recover.

Property 2: termination is checked before the model call, not after. If the budget is exhausted, calling the model wastes money and produces an output that cannot be acted on. Check first.

Property 3: the loop is bounded in four independent ways. Step count, token budget, tool-call budget, and wall-clock budget. Any one alone can fail to bound the run: an agent can make few expensive calls, or many cheap ones, or stall on a single slow tool. Four bounds are the minimum for a robust loop.

1.2.3 The tool interface

The second necessary element has two halves: the schema (what the model sees) and the dispatcher (what the harness does).

Schema. Each tool is described by a name, a natural-language description, and an argument schema. The schema must be machine-validatable. At minimum it specifies, for each argument: its type, whether it is required, and any enumerated allowed values. The schema is part of the context, so it costs tokens; §0.3 of the Prelude gives the token-cost lower bound.

Dispatcher. For each registered tool, the harness holds an executor: a function from validated arguments to a ToolResult. The dispatcher guarantees four things:

1. Validation before execution. The arguments are checked against the schema. An invalid call never reaches the executor.
2. Permission check before execution. The call is checked against the permission policy (see §1.2.5). A denied call never reaches the executor.
3. Timeout enforcement. The executor is run with a hard deadline. A slow tool cannot stall the loop indefinitely.
4. Result normalisation. Every outcome — success, error, timeout, denial — is returned as a ToolResult with a consistent shape, so that the context manager and the loop do not need to branch on the outcome type.

Tool ergonomics. The design of the schema is not cosmetic. The SWE-agent work (arXiv:2405.15793, tier R) established the agent-computer interface (ACI) thesis: at the margin, the ergonomics of the tool surface dominate the raw capability of the model. A tool that requires the model to construct an elaborate argument object will be called incorrectly more often than a tool that takes a single string. You will quantify this in Module 2’s lab; for now, note that the schema is a design decision with measurable consequences.

1.2.4 Context management

The third necessary element is a function from history to context. The minimal version has three parts.

Assembly. The context manager takes the full history and produces the ordered list of messages to be sent to the model. The minimal policy is: system prompt first, then the history in chronological order, then the current step’s instruction.

Trimming. When the assembled context exceeds a token threshold, the manager removes content. The minimal policy is oldest-first truncation of tool results, preserving the system prompt, the original user request, and the most recent k messages. The trim must be recorded in Context.dropped: what was removed, how many tokens it occupied, and the rule that removed it. This record is the raw material for Module 3’s compaction lab and for the loss analysis in §1.4.

Formatting. The manager serialises the messages into whatever format the model expects. For a proprietary model this is the vendor’s chat format; for an open-weight model it is the model’s chat template, and getting it wrong is the single most common silent failure when porting a harness (Module 13). The minimal harness should make the formatter a single replaceable function, not logic scattered through the loop.

Design principle (record it now, use it in Module 3): the context manager is the only component that may remove information from the history. Every removal is recorded. A removal that is not recorded is a silent failure, and silent failures are the ones that cost weeks.

1.2.5 Control mechanisms

The fourth necessary element has four parts in the minimal harness.

Termination conditions. The run terminates when any of the following holds: the model produces a final answer; the step limit is reached; any budget is exhausted; the loop detector fires; an unrecoverable error occurs. Each termination records a termination_reason from a fixed enumeration. The enumeration matters: Module 11’s failure taxonomy is built on it, and a harness that reports only “stopped” cannot be debugged.

Loop detection. A run is stalled if it repeats the same tool call with the same arguments more than a threshold number of times, or if its context token count fails to grow across several steps (a sign that the model is emitting the same output repeatedly). On detection, the harness injects a corrective message (“you have called read_file with the same arguments three times; the content has not changed”) and, if the stall persists, terminates with reason stalled.

Permission policy. Each tool has a permission rule. The minimal policy is a per-tool allow-list of argument patterns: for example, read_file is allowed on paths under the workspace root and denied elsewhere; write_file is allowed under the workspace root and denied elsewhere; run_command is denied unless the command is on an allow-list. The policy is enforced deterministically, in the dispatcher, before execution. It does not depend on the model complying with an instruction. This is the distinction between a guide (an instruction in the prompt) and a sensor or enforcement (a check in the harness); Module 9 develops the control-theoretic reading.

Error and retry policy. Transient failures (timeouts, rate limits) are retried with a bounded exponential backoff, at most r times. Validation failures and permission denials are not retried automatically: they are returned to the model as observations, because the correct response is for the model to change its action, not for the harness to repeat it.

1.2.6 Acceptance criteria for Lab 1a

Your implementation is accepted when it satisfies every row. Each criterion names the element it tests.

The trace requirement (#9) is not part of the four necessary elements, strictly. It is included because a harness without traces cannot be evaluated, and evaluation is the whole of Module 5. Build the trace in from the first day; retrofitting observability is far more expensive than including it.

1.2.7 Concept check

Why must every failure become an observation rather than an out-of-band log entry?

Answer. Because the model’s next action depends on its context. If a failure is logged but not placed in the context, the model proceeds as though the action succeeded, and its subsequent reasoning is built on a false premise. The failure then surfaces later as an inexplicable wrong answer, far from its cause. Making every failure an observation keeps the model’s belief — as far as the harness can — aligned with the actual state of the world.

1.3 Worked example: a three-tool harness on a real task

Task. A repository contains a Python module calc.py and a test file test_calc.py. The agent must make the test suite pass. The test currently fails because calc.add returns a string concatenation instead of an integer sum.

Registered tools.

• read_file(path: string) -> string — returns the file contents; allowed under the workspace root.
• write_file(path: string, content: string) -> string — overwrites the file; allowed under the workspace root.
• run_tests(command: string) -> {exit_code: int, stdout: string} — runs a command from an allow-list; allowed only for pytest.

Run trace (abridged).

Now perturb the run to see each control mechanism fire.

Perturbation A — malformed call. At step 3 the model emits write_file with only a path argument, omitting content. The dispatcher validates against the schema, finds the required argument missing, and produces status=error, content="missing required argument: content". This is appended to history. At the next step the model sees the error and re-emits the call correctly. The run recovers; it does not crash.

Perturbation B — denied call. At step 1 the model emits read_file("../secrets.env"). The permission policy denies it (path outside the workspace), producing status=denied. The model sees the denial and reads the in-workspace file instead. Deterministic enforcement, not instruction-following.

Perturbation C — stall. Suppose the model emits run_tests("pytest test_calc.py") three times with identical arguments, and the test keeps failing. The loop detector fires at the third repetition, injects a corrective message, and — if the next step repeats again — terminates with reason stalled. The run does not burn the entire budget on a loop.

Perturbation D — trimming. Suppose the task requires reading twenty files. At step 12 the context crosses the 8,000-token threshold. The manager drops the oldest tool results — the contents of the first eight files — and records them in Context.dropped. The model now no longer has the contents of files 1–8. If the final answer depended on file 3, the run will fail, and the trace will show exactly why: the information was dropped at step 12. This is the loss that §1.4 formalises.

Each perturbation is a test case in the Lab 1a acceptance table. Run all four before declaring the scaffold complete.

PART B — THE FORMAL FOUNDATIONS

1.4 The harness as a POMDP constructor

1.4.1 The POMDP tuple, restated

A partially observable Markov decision process is a tuple

where $\mathcal{S}$ is the state space; $\mathcal{A}$ the action space; $\mathcal{T}(s'\mid s,a)$ the transition kernel; $\mathcal{O}$ the observation space; $\mathcal{Z}(o\mid s',a)$ the observation kernel; $R(s,a)$ the reward; and $\gamma\in[0,1)$ the discount. The history is $h_t=(a_0,o_1,\dots,a_{t-1},o_t)$; the belief is $b_t(s)=\Pr(s_t=s\mid h_t)$; and the belief update is

A policy is a map $\pi:\text{histories}\to\Delta(\mathcal{A})$. (Part 0, §0.2 gives the derivations; they are not repeated here.)

1.4.2 The instantiation for a tool-using LLM agent

The tuple is abstract until instantiated. For the harness of §1.2, the instantiation is:

$\mathcal{S}$ — the state. The true state is the tuple (environment state, task state, model sampling state). The environment state includes the filesystem, the database, and any external service the tools touch. The task state is the ground-truth answer to “what still needs to be done.” The model sampling state is the random seed and temperature — it matters because it determines which of several possible outputs the model produces at each step, and therefore which trajectory the agent follows.

$\mathcal{A}$ — the action space. The set of well-formed ToolCalls, plus a distinguished finish action (the final answer). The action space is defined by the harness: the registered tools and their schemas determine which actions exist. A model cannot take an action the harness has not exposed. This is the first place the harness enters the formal object.

$\mathcal{T}$ — the transition kernel. The environment’s dynamics plus the stochastic tool layer. Reading a file does not change it (a deterministic self-transition); writing a file changes it; running a test changes nothing but observes; a network call may fail transiently. The kernel is only partly known to the harness — this is why the harness must treat tool outcomes as observations rather than as known effects.

$\mathcal{O}$ — the observation space. In a vanilla POMDP, this is the set of possible observations. In an LLM agent, the raw observation is a ToolResult — a structured record with a status and a content string.

$\mathcal{Z}$ — the observation kernel. The distribution over ToolResults given the true state and the action. A flaky test is a stochastic observation kernel; a deterministic file read is a degenerate one.

$R$ — the reward. Task success, possibly shaped by intermediate signals (tests passing, lint clean, human approval). The discount $\gamma$ is usually close to 1 for tasks that complete in tens of steps, because the reward is terminal.

So far this is a standard POMDP. The harness enters in a way that is easy to miss and decisive.

1.4.3 The decisive move: the harness defines the observation function

The model does not observe the raw ToolResult. It observes the context, which is a function of the history:

where $\phi$ is the context manager (assembly, trimming, formatting). The model’s policy is therefore

This is not a detail. It means the effective POMDP the model is solving has observation space $\mathcal{C}=\mathrm{range}(\phi)$, not $\mathcal{O}$, and observation kernel

Definition 1.4.1 (Harness-induced POMDP). Given a base POMDP $\mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{O},\mathcal{Z},R,\gamma)$ and a harness $H=(L,T,C,K)$ — loop, tool interface, context manager, control — the harness-induced POMDP is

where $\mathcal{A}_H$ is the action space defined by the registered tools, $\mathcal{T}_H$ is the environment dynamics as exposed by the dispatcher (including timeouts and denials as self-transitions), $\mathcal{O}_H$ is the space of context objects, and $\mathcal{Z}_{\phi_H}$ is the observation kernel induced by the context manager.

1.4.4 The load-bearing theorem

Theorem 1.4.2 (Harness determines return at fixed model). Let $\pi_\theta$ be a fixed model. Let $H_1$ and $H_2$ be two harnesses over the same environment. Then in general

and the difference is attributable to differences in $\mathcal{A}_H$, $\mathcal{T}_H$, and $\mathcal{Z}_{\phi_H}$.

Proof. The model is fixed, but it is a function of the context: $\pi_\theta(a\mid c)$. The return is

where the expectation is over trajectories generated by the harness-induced POMDP. The trajectory distribution depends on $\mathcal{A}_H$ (which actions are available), $\mathcal{T}_H$ (how actions change state, including timeout and denial self-transitions), and $\mathcal{Z}_{\phi_H}$ (which observations reach the model). If any of these differs between $H_1$ and $H_2$, the trajectory distribution differs, and since $R$ is non-constant, the expectation differs in general. A concrete instance: $H_1$ exposes a read_file tool; $H_2$ does not. Under $H_1$ the model can acquire the file contents and produce the correct patch; under $H_2$ it cannot, and its return is lower. The model is identical in both cases. $\square$

This theorem is the formal statement of the curriculum’s central thesis. It is almost trivial as mathematics — of course changing the action space and observation function changes the return — but it has a non-trivial methodological consequence.

1.4.5 The methodological consequence: harness-controlled comparison

Because the return depends on the harness, a comparison between two models is meaningless unless the harness is held fixed (or crossed in a factorial design). Conversely, a comparison between two harnesses is meaningless unless the model is held fixed. The first comparison is the one vendors report; the second is the one harness engineering studies. The factorial design in Part 0, §0.4 (model × harness, with the interaction term) is the only design that supports both comparisons simultaneously, and the interaction term is the quantitative measure of how much the model ranking depends on the harness.

You will run this design in Module 14 and in the capstone. The point to record now is that Theorem 1.4.2 is why the design is necessary, not merely a methodological preference.

1.4.6 Worked example: two harnesses, one model

Environment. A code repository with a failing test. Model. Fixed. Harness $H_1$: exposes read_file, write_file, run_tests; trimming keeps the last 10 messages. Harness $H_2$: same tools, but trimming keeps only the last 3 messages; and the run_tests tool returns only the exit code, not the stdout.

Run both on 200 tasks. Suppose $H_1$ succeeds on 0.71 and $H_2$ on 0.48. The model is identical. The difference decomposes into two causes, each traceable to a component:

• The aggressive trimming ($H_2$) drops the file contents read early in the run. When the model writes the patch at step 12, it no longer has the exact function signature it read at step 2. This is a lossy $\phi$: two histories with different content map to the same context, so the model’s belief is wrong.
• The exit-code-only run_tests reduces the observation space: the model cannot see which test failed, only that something failed. It must guess. This is a coarser $\mathcal{Z}_\phi$.

Neither cause is a model failure. Both are harness design failures, and both are measurable. Module 11’s trace analysis attributes the 23-point gap to these two components; Module 12’s model dossiers would have missed it entirely, because the model is not the variable.

1.4.7 Concept check

If the model is fixed, why can changing the harness change the return?

Answer. Because the model is a function of the context, and the context is produced by the harness. Changing the context manager changes the observation function, so the model sees different things and therefore takes different actions; changing the tool interface changes the action space, so some actions become available or unavailable; changing the control mechanisms changes the transition kernel, because timeouts and denials are self-transitions with consequences. All three changes alter the trajectory distribution, and therefore the expected return, with the model held fixed.

1.5 The context window as a belief state

1.5.1 The claim

The agent’s belief at step $t$ is $b_t(s)=\Pr(s_t=s\mid h_t)$. The context $c_t=\phi(h_t)$ is what the model actually conditions on. The claim of this section is: the context is a lossy, non-Bayesian surrogate for the belief, and every context-management decision is therefore a decision about how much belief information to discard.

The three adjectives matter.

Lossy. By the data-processing inequality (Part 0, §0.3): since $b_t \to h_t \to c_t$ is a Markov chain, $I(b_t;c_t)\le I(b_t;h_t)$. The context cannot carry more information about the belief than the history does, and any non-injective $\phi$ carries strictly less.

Non-Bayesian. The Bayes filter is the unique (up to null sets) belief update that is consistent with the model. The context manager is not the Bayes filter: it truncates by recency, summarises by an LLM, retrieves by embedding similarity — none of which is the posterior update. The context is a heuristic projection, and its errors are not random: they are correlated with recency, with salience, and with whatever the summariser found interesting. A model reasoning from a non-Bayesian context will have systematically distorted beliefs, not merely noisier ones.

Surrogate. It stands in for the belief. This is not a defect to be eliminated — the belief lives on a simplex and cannot be presented to a language model — but it is a defect to be managed. The management is context engineering (Module 3), and its target is to maximise $I(b_t;c_t)$ subject to the token budget.

1.5.2 The non-sufficiency result, applied

Part 0, Theorem 0.2.7 establishes that a finite context window is not a sufficient statistic for the belief: there exist POMDPs in which two histories with different beliefs map to the same truncated context. The construction is worth restating concretely because it is the mechanism behind the most common agent failure.

The scrolled-out-action construction. Suppose the agent performs an action flip at step 3 that reverses the meaning of a flag. At step 12, the agent reads the flag’s current value. If the context window has scrolled past step 3, the agent sees the flag value but not the fact that it was flipped. It cannot determine whether the value it reads corresponds to the pre-flip or post-flip semantics. Two histories — (flip at 3, read true at 12) and (no flip, read true at 12) — have different beliefs but the same truncated context. The model’s answer will be wrong for one of them, and no amount of careful reasoning can fix it, because the information is not present.

The general form. Any context policy that drops information on the basis of position (recency, oldest-first truncation) rather than on the basis of relevance to the current decision is exposed to this failure. Recency is a proxy for relevance, and it is a good proxy most of the time — which is exactly what makes it dangerous. It fails silently and rarely, on the steps where the decisive fact is old.

1.5.3 What the context manager actually loses

Three distinct losses, each with its own remedy.

Truncation loss. Content removed entirely. Remedy: retrieve it back when relevant (just-in-time loading), or externalise it to a scratchpad the model can re-read. The loss is total for the removed content; the question is whether the model can reacquire it.

Summarisation loss. Content replaced by a shorter representation. Remedy: measure the loss by downstream task success, not by summary quality (Module 3’s lab). A summary can be judged excellent and still drop the one fact the task needs.

Formatting loss. Content present but in a form the model handles poorly — for example, a JSON blob where a table would be clearer, or a long list where ordering carries meaning the formatter discarded. Remedy: design the format for the model’s attention, and test it (Module 2’s interface-efficiency metric).

All three are instances of the same formal object: a non-injective $\phi$ whose fibres contain histories with different beliefs. The remedy differs; the diagnosis is the same.

1.5.4 Concept check

Why is “the context is a belief state” a better mental model than “the context is the conversation”?

Answer. Because “the conversation” suggests that the context is simply the record of what was said, and that the only question is where to cut it off. “The belief state” makes explicit that the context is the model’s entire basis for deciding what to do, that it is a lossy projection of what the agent actually knows, and that the loss has consequences measured in task success. The belief framing forces you to ask “what does the model need to believe in order to act correctly here?” rather than “how many messages fit?”, and those are different questions with different answers.

1.6 Classical antecedents and the LLM-as-policy contrast

1.6.1 The classical definition of an agent

Wooldridge and Jennings (1995) define an agent as a system situated in an environment that is capable of autonomous action in that environment in order to meet its design objectives. They identify four properties that distinguish agents from other software:

• Autonomy: the agent operates without direct human intervention, and has control over its own internal state and behaviour.
• Reactivity: the agent perceives its environment and responds in a timely fashion to changes in it.
• Pro-activeness: the agent does not merely react but takes initiative to satisfy its objectives.
• Social ability: the agent interacts with other agents (including humans).

These four properties are a useful checklist for a harness. The loop provides reactivity; the control mechanisms provide bounded autonomy; planning (Module 4) provides pro-activeness; the tool interface provides the channel for social ability where the “other agent” is a human or a service.

1.6.2 The BDI architecture

Rao and Georgeff’s belief–desire–intention (BDI) architecture decomposes an agent into three attitudes:

• Beliefs: what the agent takes to be true about the world.
• Desires: what the agent wants to achieve (its objectives, possibly conflicting).
• Intentions: what the agent has committed to doing (a selected, adopted plan).

The BDI loop is: update beliefs from observations; select desires to pursue given beliefs; commit to intentions; execute the next action of the committed plan; repeat. The architecture’s central insight is that intentions are a commitment device: once adopted, a plan is not reconsidered at every step, which prevents thrashing.

1.6.3 The mapping to LLM agents

The mapping is not exact, and the places where it fails are instructive.

The third row is the important one. A BDI agent commits to a plan; an LLM agent typically does not, unless the harness externalises the plan. This is why Module 8 treats planning representations — task graphs, checklists, spec-driven development — as harness components rather than as model capabilities. The harness supplies the commitment device that the model lacks.

1.6.4 LLM-as-policy: what is genuinely new

Three differences distinguish an LLM agent from the classical agent, and each has a consequence for harness design.

The policy is learned from text, not programmed. The classical agent’s policy is written by an engineer; its failure modes are enumerable. The LLM’s policy is learned; its failure modes are open-ended, and the harness cannot enumerate them. This is why the harness must be a sensor-rich system: it cannot anticipate all failures, so it must detect them. The internal model principle (Part 0, §0.7) cuts both ways: the harness can only prevent failures it models, and it cannot model all of them, so detection must supplement prevention.

The belief is not maintained by a Bayes filter. The classical agent’s belief is a probability distribution updated by a known rule. The LLM’s “belief” is whatever the context implies, and the model’s reasoning over it is opaque. The harness cannot inspect the belief directly; it can only inspect the context and the output. This is why observability (Module 11) must include decision observability, not just input and output.

The action space is open-ended text, constrained by schema. A classical agent’s action space is a finite set. The LLM can emit any text; the harness constrains it to the registered tools by parsing and validation. The constraint is the harness’s job, and the quality of the constraint — the schema, the validation, the error messages — determines how often the model’s intended action survives translation into an executed action. This is the ACI thesis (§1.2.3), and it is why tool design is a first-class engineering activity rather than an afterthought.

1.6.5 Concept check

The BDI architecture commits to intentions; the typical LLM agent re-decides at every step. Is the LLM agent’s behaviour strictly worse?

Answer. No — it is a different trade-off. Re-deciding at every step allows the agent to adapt when its plan turns out to be wrong, which a committed BDI agent cannot do without an explicit reconsideration trigger. But it also allows thrashing: the agent can oscillate between plans, or abandon a good plan because a single observation was discouraging. Which is better depends on the task. For tasks where the environment is stable and the plan is likely correct, commitment is better; for tasks where observations frequently invalidate the plan, re-deciding is better. The harness can supply commitment selectively — a todo list that the agent is instructed to follow, with an explicit reconsideration rule — which is the point of Module 8.

1.7 Workflow versus agent

1.7.1 Definitions

Definition 1.7.1 (Workflow). A workflow is a system in which the control flow — the sequence of steps, the branching, the termination — is determined by the engineer at design time, and the model is called at fixed points to perform fixed subtasks. The model’s output may vary, but the path through the system does not depend on it in a way the engineer did not anticipate.

Definition 1.7.2 (Agent). An agent is a system in which the control flow is determined at run time by the model, based on observations. The model decides which tool to call next, whether to continue or stop, and how to respond to what it observes.

The distinction is about where the control flow lives, not about whether a model is involved. A retrieval-augmented generation pipeline that always retrieves, then always generates, then always returns, is a workflow even though it calls a model twice. A single-tool loop that lets the model decide when to stop is an agent even though it is small.

1.7.2 The simplest-thing-that-works principle

Anthropic’s engineering guidance (tier B) states the principle: build the simplest system that meets the requirement, and add agency only when the simpler system fails. The principle is not aesthetic. Agency has costs:

• Latency: each loop iteration is a model call.
• Cost: each iteration consumes tokens, and agent runs are long.
• Unpredictability: the model can take paths the engineer did not anticipate, including paths that fail in novel ways.
• Evaluation difficulty: a workflow’s behaviour is enumerable; an agent’s is a distribution over trajectories, which requires statistical evaluation (Module 5).
• Security surface: every tool the agent can call is an attack surface (Module 19).

A workflow that meets the requirement avoids all five. An agent should be adopted because the workflow fails, not because the agent is more impressive.

1.7.3 The formal criterion: the value of feedback

The principle can be made precise. Consider two policies over the same environment and task distribution:

• $\pi_w$: a workflow policy — an open-loop policy whose action sequence does not depend on observations. It executes a fixed plan.
• $\pi^*$: the optimal closed-loop policy, which conditions each action on the history.

The value of feedback is

If $V_f$ is small relative to the cost of agency (the latency, cost, and evaluation burden above), the workflow is the better engineering choice. If $V_f$ is large, agency is justified.

$V_f$ is large exactly when the optimal action at step $t$ depends on information that is not known at design time. That is the operational test. Ask: could I write down the correct sequence of steps now, for every task in the distribution? If yes, the workflow suffices, because the information needed to choose the sequence is already in your head. If no — because the sequence depends on what the code looks like, or which test fails, or what the user’s follow-up question is — then the sequence must be chosen at run time from observations, and that is what an agent does.

This is why the criterion is about information, not about complexity. A task can be complex and still be a workflow (a long fixed pipeline with many steps), and a task can be simple and require an agent (a two-step task whose second step depends on an observation).

1.7.4 The three-way choice

The curriculum recognises three architectures, not two. The third is multi-agent, covered in Module 18. For Module 1 you need only the decision rule at the top level:

1. Workflow if $V_f$ is small: the action sequence is knowable at design time.
2. Single agent if $V_f$ is large but the task decomposes into subtasks that share context, or if the context required fits in one window.
3. Multi-agent if $V_f$ is large and the task decomposes into subtasks that benefit from context isolation — each subtask needs a large, distinct context that would crowd out the others if shared — and the coordination cost is justified. Module 18 supplies the test; the default is single agent, because the null hypothesis (a single well-harnessed agent beats the multi-agent design) is not yet rejected in the literature.

1.8 The architecture decision procedure

The decision procedure is a decision tree. It is a hypothesis, not a law: it encodes the current evidence and should be revised as the evidence changes. Apply it, record the outcome, and revisit it when the task or the evidence changes.

1.8.1 The five axes

Each axis is a property of the task distribution, not of a single task instance.

Axis 1 — Tool count and distinctness. How many distinct tools does the task require, and are they used in a fixed order? Few tools in a fixed order suggests a workflow. Many tools in an order that varies by task suggests an agent.

Axis 2 — Observability of intermediate state. Can the next step be determined without inspecting an observation? If the engineer can write “step 1, then step 2, then step 3” without conditionals, the task is a workflow. If the next step depends on what the previous step returned, it is an agent.

Axis 3 — Verifiability of outcomes. Is the outcome checkable by a deterministic verifier (a test suite, a type checker, a schema)? This axis does not choose the architecture directly — it governs autonomy. A highly verifiable task can be run autonomously even by a simple system; an unverifiable task requires human oversight regardless of architecture. Record it because it determines the release decision (Part 0, §0.5).

Axis 4 — Cost of failure. What happens when the agent gets it wrong? A wrong draft email is cheap; a wrong database migration is expensive. High failure cost pushes toward workflows with human gates, regardless of $V_f$.

Axis 5 — Variance in task structure. How much does the structure of the task vary across the distribution? Low variance (all tasks are “fix a failing test in this repo”) supports a workflow with a fixed pipeline. High variance (tasks range from “add a feature” to “investigate a performance regression”) requires an agent, because no fixed pipeline covers the distribution.

1.8.2 The tree

Ask the questions in order. Stop at the first answer that determines the architecture.

1. Is the sequence of steps knowable at design time for every task in the distribution?
   • Yes → Workflow. Stop. (Axis 2 dominates.)
   • No → go to 2.
2. Is the cost of failure high enough that a wrong action is unacceptable without human review?
   • Yes → Workflow with human gates at the irreversible steps, or an agent restricted to reversible actions with a human gate before any irreversible one. Stop. (Axis 4 dominates.)
   • No → go to 3.
3. Does the task decompose into subtasks that each require a large, distinct context?
   • Yes, and coordination cost is justified → Multi-agent (subject to Module 18’s null-hypothesis test). Stop. (Axis 1 + context size.)
   • No → go to 4.
4. Is the task structure variance low (a small number of recognisable task types)?
   • Yes → Single agent with a task-type router, or a small set of workflows selected by a classifier. Stop. (Axis 5.)
   • No → Single agent. Stop.

The tree is deliberately biased toward the simpler architecture at every branch. That bias is correct: the simpler architecture is cheaper, faster, more evaluable, and has a smaller security surface. The burden of proof is on the more complex choice.

1.8.3 Worked application: three tasks

Task A — “Given a customer support ticket, retrieve the relevant help-centre articles and draft a reply.” Step 1: is the sequence knowable at design time? Retrieve, then draft, then return. Yes — there is no branch that depends on an observation. Workflow. No agent needed. This is the most common case, and the most commonly over-engineered.

Task B — “Make the failing tests in this repository pass.” Step 1: is the sequence knowable? No — the sequence depends on which test fails and why, which is discovered at run time. Go to 2. Step 2: is failure cost high? The tests run in a sandbox; a wrong patch is caught by the tests. Low cost. Go to 3. Step 3: does it decompose into subtasks requiring distinct large contexts? Usually not — the repo context is shared across the diagnosis, the patch, and the verification. Go to 4. Step 4: is task-structure variance low? The task type is recognisable (“fix failing tests”), but the structure varies (a typo fix vs a logic error vs a missing dependency). High variance within the type. Single agent. This is the canonical agent task, and it is why the harness matters so much: the loop, the tools, and the context manager are what make the difference between 30% and 70% success on the same model.

Task C — “Produce a competitive analysis of five companies, with a sourced section on each.” Step 1: is the sequence knowable? Partly — the five companies are known — but the research path for each depends on what is found. Go to 2. Step 2: failure cost? A wrong fact in a competitive analysis is embarrassing but not catastrophic; it is caught by review. Go to 3. Step 3: does it decompose into subtasks with distinct large contexts? Yes — each company’s research requires reading many sources, and the five contexts would crowd each other out if shared. Multi-agent, subject to the Module 18 test: build the single-agent baseline first, and only adopt multi-agent if it measurably wins on reliability, cost, and diagnosability. This is the case where multi-agent is plausible; it is not the case where it is proven.

1.8.4 The decision memo

The output of the procedure is a one-page memo with five fields:

1. Task distribution: what the system will be asked to do, and how it varies.
2. Axis values: the five axes, with a sentence of justification each.
3. Architecture chosen: workflow / single agent / multi-agent, with the tree path that produced it.
4. What would change the decision: the observation or evidence that would move the answer to a different branch. This is the hypothesis-testing discipline — it makes the decision revisable.
5. Verification plan: the verifier that will check outcomes, and its expected reliability (Part 0, §0.5).

Field 4 is the one most often omitted and the most valuable. A decision without a stated revision condition is a commitment, not an engineering judgement.

1.8.5 Concept check

A colleague proposes a multi-agent system for a task whose steps are knowable at design time. What do you say?

Answer. That the first branch of the tree already answers the question: if the steps are knowable at design time, the task is a workflow, and a multi-agent system is a more expensive, slower, less evaluable, and larger-security-surface way to execute a fixed sequence. The model calls at fixed points do not need to coordinate, because there is nothing to coordinate — the control flow is already decided. The proposal should be replaced by the workflow, and if the colleague believes the steps are not knowable, the disagreement is about Axis 2, which can be settled by examining a sample of tasks.

LABS

Lab 1a — Implement the minimal harness (Week 1)

Deliverable. A working implementation of the scaffold specified in §1.2, in your production language, plus a trace file from a three-task run.

Acceptance criteria. All ten rows of the table in §1.2.6 must pass. Each is a test; run them all and record the results.

What to submit.

1. The implementation.
2. The ten acceptance tests, with their results.
3. A trace of one successful run and one run for each of the four perturbations in §1.3 (malformed call, denied call, stall, trimming), with a short annotation of where each control mechanism fired.
4. A one-paragraph reflection: which of the four necessary elements was hardest to implement correctly, and why.

Grading note. The most common failure is criterion 5 (trimming recorded). Learners implement trimming but do not record what was dropped, which makes the loss invisible and the Module 3 lab impossible. Do not skip the record.

Lab 1b — Formal specification and architecture memo (Week 2)

Deliverable. Two documents.

Document 1: POMDP specification of your scaffold. Instantiate the tuple $(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{O},\mathcal{Z},R,\gamma)$ for the harness you built in Lab 1a, on one of the tasks you ran. For each component, give a sentence of justification. Then answer three questions in writing:

1. Name the three largest sources of partial observability in your instantiation, and classify each as environment uncertainty, tool stochasticity, or model stochasticity (§0.2 of the Prelude; restated in Part 0, §0.2.9).
2. Identify one place where your context manager is non-injective — two histories with different beliefs map to the same context — and describe the concrete task on which this causes a failure.
3. Identify one place where your action space is coarser than the task requires — a distinction the task needs that your tools cannot express — and describe the consequence.

Document 2: Architecture decision memo. Apply the decision procedure of §1.8 to three tasks from your own practice. For each, submit the five-field memo of §1.8.4. At least one of the three must result in “workflow”; if all three result in “agent”, you have either chosen unusually agentic tasks or applied the tree too permissively — re-examine Axis 2.

Acceptance criteria.

MODULE 1 CHEAT SHEET

EXERCISES WITH SOLUTIONS

E1.1. A harness exposes four tools: search, read, write, delete. The model emits read with a valid path that is outside the workspace. The permission policy denies it. Enumerate, in order, the components that act on the call, and state the status of the resulting ToolResult.

S1.1. (1) The loop parses the model output into a ToolCall. (2) The dispatcher validates the arguments against the read schema — valid. (3) The dispatcher checks the permission policy — the path is outside the workspace — denied. (4) The dispatcher produces a ToolResult with status=denied and a message naming the violated rule. (5) The loop appends the ToolResult to history. (6) The context manager includes it in the next context. The status is denied. Note that step (2) precedes step (3): validation before permission, because an invalid call should be reported as invalid, not as denied.

E1.2. A context manager trims oldest-first at 8,000 tokens. A run reads file A (2,000 tokens) at step 2, file B (2,000) at step 4, file C (2,000) at step 6, and then a long tool output (3,000) at step 8. At step 9 the context is 9,000 tokens before trimming. What is dropped, and what is the consequence if the task requires comparing A and C?

S1.2. The manager drops oldest-first until the context is under 8,000, i.e. it drops 1,000 tokens’ worth. It drops the beginning of file A’s content. The consequence: the model retains most of A but not all of it. If the comparison requires the dropped portion of A — for example a function definition that scrolled out — the model cannot perform it. The trace records the drop (Context.dropped), so the failure is attributable. Note that this is the worst kind of truncation: it leaves A looking complete when it is not, so the model has no signal that information is missing.

E1.3. Prove that if $\phi$ is injective on the set of reachable histories, then the context is a sufficient statistic for the history.

S1.3. If $\phi$ is injective on reachable histories, then from $c_t=\phi(h_t)$ the history $h_t$ can be recovered uniquely, hence $b_t$ can be computed from $c_t$ exactly as it is computed from $h_t$. Therefore $I(b_t;c_t)=I(b_t;h_t)$ and the context is sufficient. The converse does not hold: a non-injective $\phi$ can still be sufficient if it merges only histories with the same belief. This is the formal target of context engineering: find a non-injective $\phi$ that preserves $b_t$ while reducing length — a minimal sufficient statistic, which is exactly the information bottleneck problem.

E1.4. A task distribution consists of 1,000 tickets, each of which must be classified into one of 12 categories and routed to one of 4 teams. The category determines the team. Is this a workflow or an agent? Justify with the information test.

S1.4. Workflow. The sequence is knowable at design time: classify, map category to team, route. There is no step whose correctness depends on an observation gathered at run time — the classification is a single model call, and the mapping is a deterministic table. $V_f\approx0$: a closed-loop policy cannot do better than the fixed pipeline, because there is nothing to observe between the steps. The correct implementation is a single classification call followed by a lookup. This is the case most often over-engineered into an agent.

E1.5. Two harnesses, $H_1$ and $H_2$, differ only in the context manager’s trimming threshold: $H_1$ trims at 16,000 tokens, $H_2$ at 4,000. Both run the same model on the same 500 tasks. $H_1$ succeeds on 0.68, $H_2$ on 0.55. A colleague says: “the model is better in $H_1$.” What is wrong with the statement, and what experiment would settle the question?

S1.5. The statement is wrong because the model is identical in both conditions; what differs is the harness-induced POMDP. Under $H_2$, the observation function $\phi$ is more lossy, so the model operates on a worse belief surrogate. The correct statement is ”$H_1$ induces a better POMDP for this model on this task set.” The experiment that would settle which component causes the difference is a controlled ablation: vary only the trimming threshold across several values (4k, 8k, 12k, 16k) and plot success against threshold. If the curve saturates at 12k, the 16k setting is buying nothing; if it rises monotonically, the task is context-limited and the harness should invest in compaction rather than truncation.

E1.6. Apply the decision tree to this task: “Given a pull request, review it for style violations and security issues, and post a comment.” Justify each step.

S1.6. Step 1: is the sequence knowable at design time? The steps are fixed — fetch the diff, run the style checker, run the security scanner, synthesise the comment, post. The synthesis depends on the scanner outputs, but that is a fixed point of model invocation, not a run-time branching decision. Yes → Workflow. Stop. The model is called at fixed points (synthesising the comment), and the control flow is fixed. An agent would add latency, cost, and unpredictability for no gain. Note that the content of the comment depends on the scanners, but the path does not; that is the distinction.

E1.7. A harness’s loop detector fires after three identical calls. The model responds to the corrective message by making a fourth identical call. The harness terminates with reason stalled. Is this the right behaviour? What alternative design might be better?

S1.7. It is defensible: the run is not making progress, and continuing burns budget. But it is not obviously optimal. An alternative is to escalate rather than terminate: inject a stronger corrective message (“you have made this call four times; it will not produce new information; try a different approach or terminate”), and only terminate after a second threshold. The trade-off is between budget waste (escalation costs steps) and premature termination (a run that would have recovered is killed). The right design depends on the task: for cheap tasks, terminate; for expensive tasks where the model may recover, escalate. Record the choice in the decision memo’s field 4 (what would change the decision).

E1.8. Show that the value of feedback $V_f$ is zero when the observation kernel is uninformative (the observation distribution does not depend on the state).

S1.8. If $\mathcal{Z}(o\mid s,a)$ is independent of $s$, then the observation carries no information about the state: $I(s;o)=0$. By the data-processing inequality, the history carries no information about the state beyond the prior, so the belief is the prior at every step, and the optimal closed-loop policy is the optimal open-loop policy (there is nothing to condition on that is not already known). Hence $J(\pi^*)=J(\pi_w)$ and $V_f=0$. This is the formal reason a workflow suffices when the environment provides no useful feedback between steps.

EXIT TEST FOR MODULE 1

You are ready for Module 2 when you can, without notes:

1. State the four necessary elements of a harness and give the necessity argument for each.
2. Specify the minimal harness’s loop, naming the three properties that distinguish a robust loop from a fragile one.
3. Instantiate the POMDP tuple for a concrete harness, naming all seven components.
4. State and prove Theorem 1.4.2 (harness determines return at fixed model).
5. Explain why the context is a lossy, non-Bayesian belief surrogate, and give the scrolled-out-action construction.
6. Distinguish workflow from agent by where the control flow lives, and apply the information test to a task.
7. Define the value of feedback and use it to decide between a workflow and an agent.
8. Apply the five-axis decision tree to a task and write the five-field decision memo, including the revision condition.
9. Explain the ACI thesis and why tool ergonomics are a first-class design variable.
10. Explain why every failure must become an observation rather than an out-of-band log entry.

End of Module 1. Say “next” to receive Module 2 in full.