Week 23. Part IV begins. Prerequisite: Modules 1–14, especially Module 5 (pass^k, evaluation), Module 7 (verifiers), Module 10 (Harness-Zero), Module 12 (licences), Module 13 (formats), and Module 14 (distillation’s risks).

15.0 Module map and objectives

Modules 1–14 built and selected the harness and the model. Part IV turns the harness’s experience into training data: the trajectories the harness produces become the SFT set that adapts a model, and the adaptation is the loop that closes the co-adaptation (Module 14) — the model learns the harness’s conventions, and the harness’s scaffolding can shrink (Module 6, §6.13.2’s compensating components).

The module’s central discipline is that a trajectory set is a dataset, and a dataset has all the problems a dataset has: the contamination, the privacy, the licence, the distribution, and the quality. The curriculum’s evaluation discipline (Module 5) and verifier theory (Module 7) are the tools, and the module applies them to the data.

The module has five parts and one lab.

By the end of the module you will be able to do nine things:

1. Convert a harness trajectory into an SFT example, with the correct loss mask and the model’s chat template.
2. State the trajectory’s fields and the dataset’s schema, and justify each field by the training or the evaluation it supports.
3. Apply the six filters (verifier, format, length, diversity, efficiency, process) and state each filter’s purpose and cost.
4. Measure the filtered set’s effect on the student’s pass^k, not just pass@1, and explain why the distinction matters.
5. Distinguish what distils (the format, the behaviour, the knowledge) and state the sequence-level distillation’s loss.
6. Audit the dataset for contamination, privacy, and licence provenance, and produce a datasheet.
7. State the SWE-RL bridge: how the trajectory set becomes an RL environment, and what the reward is.
8. Design the SFT set’s construction as a pipeline with the Module 5 discipline (the pre-registration, the held-out validation).
9. Explain the distillation’s reward-hacking risk (Module 14, §14.14) and the audit that mitigates it.

Everything is self-contained. Where Part 0 or a prior module is used, the result is restated.

PART A — THE TRAJECTORY AS DATA

15.1 What a trajectory is

15.1.1 The structure

A trajectory is the record of a run: the sequence of the model’s outputs, the tool calls, and the observations.

Definition 15.1.1 (Trajectory). A trajectory for a task $x$ is the sequence

where $c_t$ is the context at step $t$ (the assembled prompt), $y_t$ is the model’s output (the reasoning and the tool call), $a_t$ is the parsed action, and $o_t$ is the observation (the tool’s result). The final $y_T$ is the terminating output (the answer or the final tool call).

The trajectory is the record the trace emits (Module 11, §11.1.3): the context, the output, the tool call, the result. The SFT’s data is the trajectory’s turns, and the conversion is the loss mask’s (§15.3).

15.1.2 The trajectory’s fields

The trajectory’s fields, for the training’s purposes:

The fields are the Module 11, §11.1.3’s trace’s subset, plus the training-specific fields (format_valid, teacher, task_id). The trace’s completeness is the dataset’s completeness: a trajectory without the context or the outcome is not trainable.

15.1.3 The dataset’s schema

The dataset is a collection of trajectories, with the metadata:

Definition 15.1.2 (The trajectory dataset’s schema). The dataset has:

• The trajectories: the list of the fields above.
• The split: the train/validation/test’s task identifiers, disjoint (Module 5, §5.12).
• The provenance: the teacher’s model, the harness’s version, the generation’s date, the licence.
• The filters: which filters were applied, in which order, with the counts.
• The datasheet: the contamination’s check, the privacy’s scrub, the licence’s verdict (§15.22).

The schema is the dataset’s documentation, and the datasheet is the discipline that makes the dataset reusable and auditable.

15.2 The conversation’s structure

15.2.1 The multi-turn format

The trajectory is a multi-turn conversation: the user’s task, the assistant’s thought and tool call, the tool’s result, the assistant’s next thought and call, and so on. The conversation is the model’s chat template’s (Module 13, §13.12) structure:

system:    <the harness's system prompt>
user:      <the task's instruction>
assistant: <the thought> <the tool call>
tool:      <the result>
assistant: <the thought> <the tool call>
...
assistant: <the final answer>

The conversation’s turns are the training examples’ structure, and the model’s chat template is the serialization. The template’s correctness is the format test’s (Module 13, §13.12.3): a trajectory serialized with the wrong template trains the model on the wrong format.

15.2.2 The tool’s role

The tool’s result is the observation, and it is not the model’s output. In the loss mask (§15.3), the observation’s tokens are masked: the model is trained to predict its own outputs (the thoughts and the calls), not the environment’s observations. The distinction is the whose-tokens question: the model’s tokens are the target, and the environment’s are the context.

Design principle (the whose-tokens rule). The loss mask is 1 for the assistant’s tokens (the thoughts and the tool calls) and 0 for the system’s, the user’s, and the tool’s tokens. A mask that includes the observation’s tokens trains the model to predict the environment, which is not the task.

The rule’s rationale: the model’s task is to produce the next action given the observation, not to produce the observation. Training on the observation’s tokens wastes the gradient and can teach the model to hallucinate the results (a failure that is invisible in the training’s loss but visible in the deployment).

15.2.3 The format’s target

The tool call’s serialization is the format’s target: the model must learn the harness’s tool-call format (the JSON’s schema, the function-call’s syntax). The target is the native template’s (Module 13, §13.15), and the trajectory’s tool-call field is the target’s content. The SFT’s format’s learning is the distillation’s most reliable transfer (§15.15).

15.3 The loss mask

15.3.1 The formula

Definition 15.3.1 (The SFT loss with a mask). For a token sequence $t_1,\dots,t_n$ and a mask $m_i\in{0,1}$, the SFT loss is

The mask’s normalization (the denominator) is the number of the unmasked tokens, so the loss is the per-target-token’s average, not the per-sequence’s. The normalization matters: a sequence-level normalization would weight the long sequences more, and the short sequences (the tool calls) less, which is the wrong weighting when the tool calls are the task’s core.

15.3.2 The mask’s construction

The mask is constructed by the turn’s role:

1. Serialize the conversation with the model’s chat template.
2. Identify the assistant’s spans (the thought’s and the tool call’s tokens).
3. Set $m_i=1$ for the assistant’s spans, $m_i=0$ elsewhere.

The construction’s subtlety: the template’s special tokens (the role markers) are the template’s, not the model’s output. The model’s output starts after the assistant’s role marker. A mask that includes the role marker trains the model to produce the marker, which the harness’s formatter already produces — a waste, and a potential format’s conflict.

Test 15.3.2 (The mask’s test). Serialize a known trajectory, and assert: (1) the masked tokens are exactly the assistant’s content; (2) the role markers, the tool results, and the system prompt are masked; (3) the masked token count equals the sum of the assistant’s output lengths.

The test is the dataset’s CI, and it catches the mask’s errors before the training’s cost.

15.3.3 The masked loss’s properties

Property 1 — The gradient’s focus. The mask focuses the gradient on the model’s decisions (the thought and the call), which is the task’s content.

Property 2 — The observation’s exclusion. The observation’s exclusion prevents the model from learning to predict the tool’s output, which is the environment’s, not the model’s.

Property 3 — The format’s learning. The assistant’s spans include the tool-call’s syntax, so the model learns the format from the target’s tokens.

15.3.4 Concept check

Why is masking the observation’s tokens necessary, and what would happen without the mask?

Answer. The observation is the environment’s output (the tool’s result), not the model’s decision. Without the mask, the model is trained to predict the observation’s tokens given the context, which teaches it to hallucinate the tool’s results (the model learns that after a tool call, the “expected” result is the recorded one, and it can produce the result without the tool). The training’s loss would improve (the observation is predictable from the trajectory), and the deployment’s behaviour would degrade (the model produces the results instead of calling the tools). The mask’s exclusion is the discipline that keeps the model’s learning on the task.

15.4 The data’s quality

15.4.1 The quality’s dimensions

A trajectory’s quality has five dimensions, and each is a filter’s or a metric’s target.

Dimension 1 — The outcome’s correctness. The trajectory’s verifier’s verdict. The dimension is the outcome’s quality, and it is the verifier’s precision’s (Module 7, §7.8).

Dimension 2 — The format’s validity. The trajectory’s outputs parse into the harness’s format. The dimension is the format’s quality, and it is the parse-failure rate’s complement.

Dimension 3 — The process’s coherence. The trajectory’s reasoning is coherent (no repetition, no contradiction, no stall). The dimension is the process’s quality, and it is the loop detector’s and the reasoning’s analysis’s (Module 4, §4.1).

Dimension 4 — The efficiency. The trajectory’s steps and tokens are within the budget. The dimension is the cost’s quality, and it is the three-term invariant’s (Part 0, §0.9).

Dimension 5 — The diversity. The trajectories cover the task distribution, not a few tasks repeatedly. The dimension is the coverage’s quality, and it is the deduplication’s and the stratification’s (Module 5, §5.11).

15.4.2 The quality’s measurement

Each dimension is measurable:

The thresholds are the filters’ parameters (§15.8), and the measurement is the filters’ input.

15.5 The data’s distribution

15.5.1 The distribution’s shift

The trajectory set’s distribution is the teacher harness’s task distribution, which may differ from the student’s deployment distribution. The shift has three sources.

Source 1 — The task’s selection. The teacher’s tasks are the ones the teacher was run on, which may be the easy ones (the teacher’s success’s bias) or the ones the harness’s team chose.

Source 2 — The harness’s version. The teacher’s harness is the version at the generation’s time, which may differ from the deployment’s harness (the tools, the prompt, the format).

Source 3 — The model’s version. The teacher’s model is the version at the generation’s time, which may differ from the deployment’s model.

The shift’s consequence: the student is trained on the teacher’s distribution, and the deployment’s distribution is the student’s test. The held-out family (Module 5, §5.11.2) is the check, and the datasheet’s distribution’s description is the documentation.

15.5.2 The stratification

The set’s stratification is the distribution’s control: the trajectories are sampled by the task’s type, the difficulty, and the source, so the set’s distribution matches the deployment’s (as closely as the teacher’s data allows).

Definition 15.5.1 (The stratified set). A stratified set samples the trajectories by the strata (the task’s type, the difficulty’s bucket, the source), with the counts proportional to the deployment’s distribution (or the target’s).

The stratification’s benefit: the student’s performance is balanced across the strata, not dominated by the most common one. The stratification’s cost: the rare strata are under-sampled (the teacher’s data may not have enough), and the set’s size is the binding constraint.

15.6 The dataset’s size

15.6.1 The size’s question

The dataset’s size is the data’s scaling question: more trajectories improve the student, up to a point, and the point depends on the task, the model, and the filters.

The empirical picture (the general SFT literature, tier P/R): the improvement is steep for the first few thousand trajectories, and flattens after; the flat point depends on the task’s diversity (a diverse task needs more data) and the model’s capacity (a smaller model saturates earlier).

15.6.2 The lab’s size

The lab’s size (5,000 trajectories) is the curriculum’s choice, and its rationale is:

• The steep region’s coverage: 5,000 is in the steep-to-flat transition for the typical coding agent’s tasks.
• The filters’ cost: the filters reduce the 5,000 to the usable set, and the usable set must be large enough for the training.
• The measurement’s power: the pass^k’s comparison needs the student’s evaluation on a held-out set, and the 5,000’s set is large enough to leave a held-out set (the split’s 80/20, or 90/10).

The size is a parameter, and the lab’s result is the size’s effect (the learning curve’s shape), not the size’s optimum.

PART B — REJECTION SAMPLING AND FILTERING

15.7 The rejection sampling

15.7.1 The mechanism

Rejection sampling generates $n$ trajectories per task and keeps the ones that pass the verifier.

Definition 15.7.1 (Rejection sampling). For a task $x$, generate $n$ trajectories $\tau_1,\dots,\tau_n\sim\pi_{\text{teacher}}$, and keep the subset $S_x={\tau_i: V(\tau_i)=1}$, where $V$ is the verifier. The set’s size is $|S_x|$, and the acceptance rate is $\mathbb{E}[|S_x|]/n$.

The mechanism is the Module 4, §4.6’s best-of-n, at the data’s level: the verifier selects the successful trajectories, and the selection’s precision is the verifier’s precision (Part 0, §0.5).

15.7.2 The acceptance rate’s determinants

The acceptance rate depends on three factors.

The teacher’s per-task success $p$. The expected acceptance rate is $p$ (for a perfect verifier), so a stronger teacher has a higher rate and a cheaper dataset.

The verifier’s reliability. The imperfect verifier accepts some failures (the false accepts) and rejects some successes (the false rejects). The acceptance rate is $\mathrm{TPR}\cdot p+\mathrm{FPR}\cdot(1-p)$ (the acceptance’s formula, Part 0, §0.5).

The sampling’s $n$. The expected number of accepted trajectories is $n\times$ the acceptance rate, so a larger $n$ yields more data at a linear cost.

15.7.3 The cost’s arithmetic

The dataset’s cost is the teacher’s generation cost plus the verifier’s cost, per accepted trajectory:

Worked example. $C_g=0.05$, $C_v=0.01$, the acceptance rate 0.4: $C_{\text{per accepted}}=(0.05+0.01)/0.4=0.15$. To build 5,000 trajectories: $5{,}000\times0.15=750$ in cost units. The lesson: the acceptance rate is the dataset’s cost multiplier, and a stronger teacher (a higher rate) is cheaper per trajectory.

15.7.4 The sampling’s $n$

The $n$‘s choice is the diversity’s trade-off: a larger $n$ yields more trajectories per task, but the trajectories are correlated (the same task’s samples share the task’s content). The set’s diversity is the number of the unique tasks, not the total trajectories. A set with 5,000 trajectories from 100 tasks is less diverse than one with 5,000 from 5,000 tasks, even though the size is the same.

Design principle (the diversity’s floor). The dataset’s unique-task count is the diversity’s measure, and the set’s size’s value is the unique tasks’ count, not the trajectories’ count. The $n$‘s choice should be bounded by the diversity’s floor (e.g., $n\le5$ per task), not by the generation’s budget.

15.8 The six filters

15.8.1 The filters

The trajectories are filtered by six criteria, applied in order.

Filter 1 — The verifier’s pass. Keep the trajectories the verifier accepts. The filter’s purpose: the outcome’s correctness. The cost: the verifier’s run (the $C_v$).

Filter 2 — The format’s validity. Keep the trajectories whose outputs parse. The filter’s purpose: the format’s quality. The cost: the parse’s check (cheap).

Filter 3 — The length’s bound. Keep the trajectories within the max steps and the max tokens. The filter’s purpose: the efficiency’s and the context’s fit. The cost: the trace’s inspection (cheap).

Filter 4 — The diversity’s deduplication. Remove the near-duplicate trajectories (the same task’s repeated samples, the same trajectory’s paraphrase). The filter’s purpose: the coverage’s quality. The cost: the similarity’s computation (moderate).

Filter 5 — The process’s coherence. Remove the trajectories with the repetition, the stall, the contradiction. The filter’s purpose: the process’s quality. The cost: the loop detector’s and the reasoning’s analysis (moderate).

Filter 6 — The efficiency’s budget. Keep the trajectories within the token budget (the cost’s ceiling). The filter’s purpose: the cost’s quality. The cost: the trace’s sum (cheap).

The six filters are the five quality dimensions’ (§15.4) plus the diversity’s, and each filter’s cost is the price of its dimension.

15.8.2 The filters’ order

The filters’ order is the cost’s order: the cheap filters first, the expensive ones on the survivors.

1. The verifier’s pass (the most expensive, but it is the primary’s quality — it runs first because it is the outcome’s gate).
2. The format’s validity (cheap).
3. The length’s bound (cheap).
4. The efficiency’s budget (cheap).
5. The process’s coherence (moderate).
6. The diversity’s deduplication (moderate, and it depends on the set’s final composition).

The order’s rationale: the verifier’s pass is the primary filter (the outcome is the task’s success), and the others are the secondary (they refine the primary’s set). The order’s cost: the verifier’s run is the expensive one, and it runs first because the secondary filters’ value is on the verified set.

15.8.3 The filters’ interaction

The filters are not independent: the verifier’s pass and the format’s validity are correlated (a format failure is often a failure), and the diversity’s deduplication and the efficiency’s budget are anti-correlated (a longer trajectory is more informative, and a shorter is cheaper). The interaction is the lab’s subject (§15.29), and the analysis is the filters’ marginal effect, not the individual’s.

15.9 The verifier’s filter

15.9.1 The filter’s precision

The verifier’s filter’s quality is the verifier’s precision (Part 0, §0.5): the fraction of the accepted trajectories that are actually correct. A verifier with a high false-accept rate admits the wrong trajectories, and the student learns the wrong behaviour (the reward hacking’s risk, Module 14, §14.14).

Proposition 15.9.1 (The filter’s precision and the student’s quality). If the verifier’s precision is $\pi_v$, the filtered set contains a fraction $1-\pi_v$ of the wrong trajectories. The student’s quality is bounded by the set’s quality: the student learns the wrong trajectories’ behaviour, and its success rate on the deployment is at most the set’s quality (plus the student’s generalisation).

Argument. The SFT’s objective is the likelihood of the filtered set; the wrong trajectories are in the set, so the student’s likelihood of the wrong behaviour is increased. The bound is the set’s quality, and the student’s success cannot exceed it (on the tasks the wrong trajectories cover). $\square$

The proposition is the reason the verifier’s calibration (Module 7, §7.8.3) is the dataset’s first requirement: an uncalibrated verifier is an uncalibrated dataset.

15.9.2 The filter’s audit

The filter’s audit is the Module 5, §5.13.3’s: the exploit list, the exploit rate, the corrected score. The audit’s target: the filtered set’s false accepts. The procedure:

1. Sample the filtered set’s trajectories.
2. Re-verify them with a stronger verifier (the held-out suite, the human’s review).
3. Count the false accepts.
4. Correct the set (remove the false accepts) or the verifier (improve its precision).

The audit is the set’s quality’s check, and it is the lab’s part.

15.10 The format’s filter

15.10.1 The filter’s purpose

The format’s filter removes the trajectories whose outputs do not parse. The purpose: the student learns the valid format, not the invalid one.

The filter’s subtlety: a trajectory can have some valid and some invalid outputs. The filter’s choice:

• The strict filter: keep the trajectory only if all the outputs parse.
• The step’s filter: keep the valid steps, and drop the invalid ones (the trajectory is split into the valid steps).

The strict filter’s rationale: a trajectory with an invalid step is a trajectory whose behaviour is unreliable (the model’s format is inconsistent). The step’s filter’s rationale: the valid steps are the useful data, and the invalid steps are the noise. The choice is the data’s quantity vs the quality’s trade-off, and the lab’s comparison is the answer.

15.10.2 The filter’s effect

The format’s filter’s effect is the parse-failure rate’s reduction in the student (Module 12, §12.12). The filter’s mechanism: the student learns from the valid examples, so its format’s compliance improves.

The worked expectation. The teacher’s parse-failure rate is 10%; the format’s filter keeps the valid trajectories; the student’s parse-failure rate is the set’s rate (0%, if the strict filter) plus the student’s generalisation’s gap (the student’s format’s errors). The lab measures the gap.

15.11 The length’s and the efficiency’s filters

15.11.1 The length’s filter

The length’s filter keeps the trajectories within the max steps and the max tokens. The purpose: the context’s fit (the trajectory must fit the student’s context, Module 3) and the efficiency’s (the long trajectories are the cost’s outliers).

The filter’s risk: the long trajectories are the hard tasks’ (the tasks that need the exploration), and the filter removes them, biasing the set toward the easy tasks. The mitigation: the filter’s bound should be the student’s context’s (the effective length), not an arbitrary number, and the set’s distribution should be checked after the filter.

15.11.2 The efficiency’s filter

The efficiency’s filter keeps the trajectories within the token budget. The purpose: the cost’s control (the long trajectories are the expensive ones).

The filter’s risk: the same as the length’s (the long trajectories are the hard tasks’). The mitigation: the budget should be the deployment’s cost’s requirement, and the filter’s effect on the distribution should be measured.

15.11.3 The filters’ combined effect

The length’s and the efficiency’s filters are correlated (a long trajectory is also a token-expensive one), and their combined effect is the set’s truncation (the tail’s removal). The truncation’s cost: the hard tasks are under-represented, and the student’s performance on the hard tasks is worse than on the easy ones.

The measurement: the set’s distribution before and after the filters, and the student’s performance by the task’s difficulty. The lab’s stratification (Module 5) is the measurement.

15.12 The diversity’s filter

15.12.1 The deduplication

The diversity’s filter removes the near-duplicate trajectories:

• The task’s duplicates: the same task’s multiple accepted samples. The keep: one per task (or $k$ per task, with the $k$‘s diversity).
• The trajectory’s duplicates: the near-identical trajectories (the same steps, the same outputs). The similarity’s measure: the tool-call sequence’s similarity (Module 11, §11.4.1’s clustering), or the output’s embedding’s similarity.

15.12.2 The diversity’s measure

Definition 15.12.1 (The set’s diversity). The set’s diversity is

or a similar composite. The first factor is the task’s coverage; the second is the trajectory’s variety within the task.

The diversity’s floor: the set’s task-type coverage should match the deployment’s (the stratified set, §15.5.2), and the trajectory’s variety should be enough that the student learns the task’s range of solutions, not a single one.

15.12.3 The diversity’s effect on pass^k

The diversity’s effect is the pass^k’s (Module 5, §5.5): a set with the duplicated tasks overfits to those tasks, and the student’s pass^k (the reliability on the unseen tasks) is lower than its pass@1 suggests. The mechanism: the duplicates teach the student the specific tasks’ solutions, and the student’s generalisation to the new tasks is worse.

Proposition 15.12.2 (The duplicates’ pass^k’s penalty). A set with the fraction $f$ of the duplicates has the effective sample size $N_{\text{eff}}\approx N(1-f)$ (the unique tasks’ count), and the student’s pass^k on the unseen tasks is lower than the set’s size would suggest, by the generalisation’s gap that the duplicates’ overfitting creates.

Argument. The duplicates add no new information (the same task’s solution), so the effective sample size is the unique tasks’ count. The student’s overfitting to the duplicated tasks is the variance’s reduction on the training’s tasks and the bias’s increase on the unseen ones. The pass^k’s measure (the reliability) is the unseen tasks’, and the duplicates’ penalty is the generalisation’s gap. $\square$

The proposition is the reason the diversity’s filter is the pass^k’s most important filter, and the lab’s key hypothesis (H2).

15.13 The process’s filter

15.13.1 The coherence’s check

The process’s filter removes the trajectories with the incoherent process: the repetition (the same call, the same output), the stall (the no-progress), the contradiction (the reasoning that contradicts the observation), and the loop (the cycle).

The check’s mechanism: the loop detector’s flags (Module 1, §1.2.5), the repetition’s count, the reasoning’s analysis (the contradiction’s detection, a model’s or a rule’s check).

15.13.2 The filter’s risk

The process’s filter’s risk: the long but legitimate trajectories (the exploration, the retries that eventually succeed) are flagged as the repetition or the stall. The filter’s threshold should distinguish the useful repetition (the polling with the state’s change) from the useless one (the same call with no change), as the Module 7, §7.21.3’s loop detector does.

15.13.3 The filter’s effect

The process’s filter’s effect: the student learns the coherent process, not the stalling one. The effect’s measure: the student’s repetition’s rate, its stall’s rate, and its pass^k (the coherent process is more reliable).

PART C — TEACHER–STUDENT DISTILLATION

15.14 The pipeline

15.14.1 The five steps

The distillation’s pipeline (restating Module 14, §14.13.1, with the data’s detail):

Definition 15.14.1 (The distillation pipeline).

1. Generate. The teacher (the strong model + the harness) runs on the task set, producing the trajectories.
2. Filter. The six filters (§15.8) reduce the trajectories to the training set.
3. Convert. The trajectories are converted to the SFT examples (the conversation’s structure, the loss mask, the chat template).
4. Train. The student is trained (the SFT’s loss, §15.3).
5. Evaluate. The student’s pass^k, the parse rate, the cost per correct task, and the exploit’s rate.

The pipeline is the data’s engineering, and each step’s discipline is the module’s subject.

15.14.2 The conversion’s detail

The conversion’s steps:

1. The task’s context: the system prompt, the user’s message.
2. The turns: for each step, the assistant’s output and the tool’s result.
3. The chat template: the model’s (the student’s, not the teacher’s — the student’s format’s target).
4. The mask: the assistant’s spans (the thought and the call), the 0 elsewhere.
5. The tool-call’s serialization: the student’s native format’s (the format’s target).

The conversion’s subtlety: the teacher’s format may differ from the student’s (a different tool-call’s syntax). The conversion must re-serialize the teacher’s tool calls into the student’s format, or the student learns the teacher’s format, which the student’s harness’s parser does not expect.

Design principle (the re-serialization). The trajectory’s tool calls are re-serialized into the student’s native format, not the teacher’s. The teacher’s content (the call’s name and arguments) is preserved, and the syntax is the student’s.

The re-serialization’s test: the student’s format test (Module 13, §13.16.1) on the converted examples.

15.15 What transfers

15.15.1 The three transfers

Restating Module 14, §14.13.2, with the evidence:

The format’s transfer. The student learns the tool-call’s syntax, the reasoning’s format, the stop tokens. The transfer is the most reliable: the SFT’s pattern’s learning is the format’s, and the student’s format’s compliance is high. The evidence: the parse-failure rate’s reduction (the lab’s measurement).

The behaviour’s transfer. The student learns the strategy (the decomposition, the tool’s selection, the error’s recovery). The transfer is partial: the common patterns are learned, and the rare ones are not. The evidence: the success rate’s improvement, and the pass^k’s (the reliability’s) gap.

The knowledge’s transfer. The student learns the domain’s knowledge (the code’s conventions, the tool’s semantics). The transfer is the least reliable: the student’s capacity is smaller, and the knowledge is compressed lossily. The evidence: the domain-specific tasks’ performance.

15.15.2 The transfer’s measurement

The three transfers are measurable by the ablations:

• The format’s ablation: the student trained with the format’s examples vs without (the format’s examples are the tool-call’s syntax). The difference is the format’s transfer.
• The behaviour’s ablation: the student trained on the successful trajectories vs the mixed. The difference is the behaviour’s transfer.
• The knowledge’s ablation: the student trained on the domain’s tasks vs a generic set. The difference is the knowledge’s transfer.

The ablations are the lab’s variants, and the analysis is the transfer’s decomposition.

15.15.3 Concept check

Why is the format’s transfer more reliable than the knowledge’s?

Answer. Because the format is a pattern (the tool-call’s syntax, the stop tokens), and the SFT’s pattern’s learning is reliable: the pattern is consistent across the examples, and the student’s capacity is sufficient to learn it. The knowledge is a distribution (the code’s conventions, the tool’s semantics), and the student’s capacity is smaller, so the knowledge is compressed lossily. The format’s transfer is the harness’s compensating component’s removal (Module 6, §6.13.2), and the knowledge’s transfer is the model’s capability’s upgrade — the first is the harness’s win, the second is the model’s.

15.16 The sequence-level loss

15.16.1 The loss’s form

The distillation’s loss is the SFT’s loss on the teacher’s trajectories (the sequence-level distillation), or the KL’s to the teacher’s distribution (the logit-level distillation).

The sequence-level (the SFT’s): the loss is $-\sum_i m_i\log\pi_\theta(y_i)$, where $y$ is the teacher’s output. The student learns the teacher’s samples (the sequences).

The logit-level: the loss is $D_{\mathrm{KL}}(\pi_{\text{teacher}}|\pi_\theta)$ on the same context. The student learns the teacher’s distribution (the probabilities).

15.16.2 The choice

The logit-level is more informative (the distribution has more information than the sample), and it requires the teacher’s logits, which the API’s models do not provide (the closed API gives the samples, not the logits). The open-weight teacher (the local model) provides the logits, and the logit-level is available.

The choice’s decision:

• The closed teacher (the API): the sequence-level (the SFT’s). The information is the samples.
• The open teacher (the local): the logit-level (the KL’s). The information is the distribution, and the student’s learning is more efficient.

The lab’s teacher is the harness’s model, which may be closed or open; the loss’s choice is the teacher’s availability’s.

15.16.3 The loss’s properties

Property 1 — The mask’s role. The mask (the assistant’s spans) is the loss’s target, and the observation’s exclusion is the same as the SFT’s.

Property 2 — The temperature. The logit-level’s KL is often computed at a temperature $T>1$ (the soft targets, Hinton’s distillation), which smooths the distribution and reveals the teacher’s uncertainty. The sequence-level has no temperature (the sample is the target).

Property 3 — The capacity’s gap. The student’s capacity is smaller, and the loss’s optimum is the student’s capacity’s best approximation of the teacher, not the teacher’s exact replication. The loss’s over-fitting to the teacher’s samples is the capacity’s gap’s risk, and the held-out’s validation is the check.

15.17 The evidence

15.17.1 The distillation’s evidence

The distillation’s evidence (the general literature, tier P/R):

• The format’s transfer is reliable (the SFT’s pattern’s learning).
• The behaviour’s transfer is partial (the success’s improvement, the pass^k’s gap).
• The knowledge’s transfer is the weakest (the capacity’s limit).

The Harness-Zero (Module 10, §10.11, tier P) is the specific claim: the harness’s behaviour distils into the student, and the student needs less scaffolding. The claim’s status is P (unreplicated), and the lab’s measurement is the replication’s attempt.

15.17.2 The lab’s role

The lab’s role is the measurement: the 5,000-trajectory set, the filters’ analysis, and the student’s pass^k. The lab does not prove the distillation’s general claim; it measures the claim on the learner’s task, and the result is the learner’s own evidence (Module 12, §12.11’s reproduction’s discipline, applied to the distillation).

15.18 The reward-hacking’s risk

15.18.1 The mechanism

Restating Module 14, §14.14.1: the teacher’s successful trajectories may include the verifier’s exploits, and the student learns them. The distillation’s success is inflated by the exploit’s rate, and the student’s true performance is lower.

Proposition 15.18.1 (The filter’s precision and the exploit’s amplification). If the verifier’s precision is $\pi_v$, the filtered set’s exploit’s fraction is $1-\pi_v$, and the student’s exploit’s rate is at least $1-\pi_v$. The SFT reinforces the exploit’s pattern (the short, high-reward sequences), so the student’s rate is often higher.

Argument. The exploit’s trajectories are in the set (the verifier accepted them), so the SFT’s likelihood of the exploit’s pattern is increased. The pattern’s short length and high reward make it a strong gradient’s signal, so the student’s rate is at least the set’s, and often higher. $\square$

15.18.2 The audit

The audit is the Module 5, §5.13.3’s and Module 14, §14.14.2’s: the teacher’s exploit’s rate, the student’s exploit’s rate, the corrected score. The dataset’s audit is the first gate (the teacher’s trajectories), and the student’s audit is the final gate (the deployment’s).

15.18.3 The mitigation

The mitigation’s options:

• The verifier’s precision’s improvement (a stronger verifier, Module 17b’s calibration).
• The set’s filtering (the audit’s removal of the false accepts).
• The student’s post-training’s audit (the student’s exploit’s rate’s measurement, and the removal or the penalty).

The mitigation’s cost is the verifier’s, and the benefit is the student’s true performance. The decision is the cost-benefit’s (the Module 12, §12.16’s gate’s order).

PART D — DATA ENGINEERING

15.19 Contamination

15.19.1 The mechanism

Contamination in the trajectory set is the presence of the evaluation’s tasks (the test set) in the training’s data. The teacher’s tasks and the student’s test’s tasks must be disjoint (the split, §15.1.3), and the contamination’s sources are:

• The task’s overlap: the teacher’s tasks and the test’s tasks are the same (the split’s violation).
• The leakage: the test’s content is in the teacher’s tasks (the test’s task’s description, the solution).
• The model’s training: the teacher’s model was trained on the test’s content (the base model’s contamination, Module 5, §5.11).

15.19.2 The detection

The detection’s methods (Module 5, §5.11.2):

• The split’s check: the task’s identifiers are disjoint.
• The overlap’s check: the n-gram’s or the embedding’s similarity between the train and the test’s tasks.
• The canary: the unique tokens in the test’s tasks, and the check whether the student’s outputs reproduce them.

15.19.3 The control

The control: the held-out family (Module 5, §5.11.2), the non-public tasks from the deployment’s distribution, and the split’s discipline (the test is used once, after the training’s completion).

The datasheet’s contamination’s field (§15.22) is the record, and it is the dataset’s reusability’s condition.

15.20 Privacy

15.20.1 The personal data

The trajectories may contain the personal data: the users’ content, the names, the emails, the credentials, the health’s data. The training’s set is the data’s processing, and the privacy’s laws (the GDPR, the sectoral rules) apply.

The privacy’s controls:

• The scrubbing: the automated detection and the removal of the personal data (the PII’s detectors, the regex’s, the model’s classifiers).
• The minimization: the set’s retention of the only necessary data (the task’s content, not the user’s identity).
• The consent: the data’s subjects’ consent (the user’s agreement to the training’s use).
• The access’s control: the set’s storage and the access’s audit (Module 11, §11.6.3).

15.20.2 The secrets

The trajectories may contain the secrets: the API keys, the passwords, the tokens. The secret’s presence in the training’s set is the credential’s leak, and the model’s memorization can reproduce them (the extraction’s attacks).

The controls:

• The secret’s scanning: the regex’s and the entropy’s detectors (the sk-, the AKIA, the high-entropy strings).
• The rotation: the leaked secrets’ rotation (the leak’s containment).
• The redaction: the secret’s replacement with the placeholder, not the removal (the context’s structure’s preservation).

15.20.3 The privacy’s measurement

The privacy’s measurement is the audit: the sample of the set’s trajectories, the PII’s and the secret’s detection, and the false-negative’s rate (the undetected data). The audit’s result is the datasheet’s privacy’s field, and the residual’s risk is the deployment’s acceptance.

15.21 Licence provenance

15.21.1 The three licences

The trajectory set’s licence has three components (Module 12, §12.7’s per-checkpoint’s rule, applied to the data):

The teacher’s licence. The model that generated the trajectories. A Class 3 (non-commercial) teacher’s trajectories are the teacher’s outputs, and the licence’s terms may restrict the derivative’s use. A Class 2’s terms (the acceptable-use, the pass-through) follow the outputs, and the student’s deployment must comply.

The data’s licence. The tasks’ content (the code, the documents) has its own licence (the open-source’s, the proprietary’s). The set’s use of the data is the licence’s question, and a proprietary code’s trajectory cannot be used without the owner’s permission.

The derivative’s licence. The student’s weights are the teacher’s derivative (the distillation’s), and the student’s licence is the teacher’s (the Class 2’s pass-through, Module 12, §12.7.2). The student’s deployment must comply with the teacher’s conditions.

15.21.2 The verdict

The licence’s verdict is the four-field’s (Module 12, §12.7.3): the checkpoint (the teacher and the student), the class, the conditions, the verdict. The dataset’s datasheet records the verdict, and the student’s deployment’s gate is the verdict’s.

15.21.3 The provenance’s record

The provenance’s record is the dataset’s metadata: the teacher’s model and version, the harness’s version, the generation’s date, the licence’s URLs, the filters’ history. The record is the dataset’s reproducibility’s and the licence’s audit’s basis.

15.22 The datasheet

15.22.1 The datasheet’s sections

The datasheet (the dataset’s documentation, following the “Datasheets for Datasets” framework, Gebru et al., 2021, tier R) has the sections:

1. The motivation. Why the dataset was created, and for what task.

2. The composition. The trajectories’ count, the tasks’ count, the fields, the format, the distribution (the task types, the difficulty, the source).

3. The collection. The teacher’s model, the harness’s version, the generation’s process, the acceptance’s rate, the filters (the six, with the counts).

4. The preprocessing. The conversion (the mask, the template, the re-serialization), the scrubbing (the PII, the secrets), the deduplication.

5. The uses. The intended use (the SFT), the prohibited uses (the deployment’s beyond the licence), the evaluation’s caveats.

6. The distribution’s shift. The teacher’s distribution vs the deployment’s, the held-out family’s definition.

7. The maintenance. The versioning, the updates, the retention.

8. The ethics. The privacy’s audit, the contamination’s check, the licence’s verdict.

15.22.2 The datasheet’s role

The datasheet is the dataset’s reusability’s condition: a dataset without the datasheet is a dataset of unknown provenance, and the licence’s audit and the contamination’s check are impossible. The datasheet is the discipline that makes the trajectory set a dataset, not a pile of the logs.

15.22.3 The datasheet’s cost

The datasheet’s cost is the documentation’s time, and its benefit is the reuse’s and the audit’s. The cost-benefit’s is the same as the Module 12, §12.9’s dossier’s: the documentation is the discipline that prevents the expensive mistakes (the licence’s violation, the contamination’s invalid evaluation).

15.23 Concept check

A trajectory set has no datasheet, and the team wants to use it for the SFT. What are the four questions to ask before the training, and why?

Answer. (1) The licence’s provenance: the teacher’s, the data’s, and the derivative’s licences — without them, the student’s deployment may be a violation. (2) The contamination’s check: whether the test’s tasks are in the set — without it, the evaluation is invalid. (3) The privacy’s audit: whether the set contains the PII or the secrets — without it, the training is a privacy’s violation and the model can leak. (4) The distribution’s shift: whether the teacher’s tasks match the deployment’s — without it, the student’s performance is the teacher’s distribution’s, not the deployment’s. The four questions are the datasheet’s core, and the dataset without them is the dataset of the unknown’s risk.

PART E — SWE-AGENT-STYLE DATASETS AND SWE-RL

15.24 The SWE-agent’s trajectory’s datasets

15.24.1 The dataset’s form

The SWE-agent’s work (arXiv:2405.15793, tier R) produced the trajectory’s datasets: the agent’s runs on the repository’s tasks, with the tool calls, the observations, and the outcomes. The dataset’s form is the multi-turn’s conversation (§15.2), with the repository’s tools (the edit, the search, the read).

The dataset’s use: the SFT’s data (the agent’s behaviour’s learning), and the analysis’s data (the ACI thesis’s, Module 2, §2.3).

15.24.2 The dataset’s characteristics

The SWE-agent’s dataset’s characteristics:

• The task’s source: the GitHub’s issues, with the repository’s state.
• The tools: the SWE-agent’s ACI (the edit, the search, the read, the run).
• The outcome: the test’s pass/fail (the verifier).
• The trajectory’s length: the tens of steps, the thousands of tokens.

The dataset’s role in the curriculum: the reference’s dataset, and the lab’s template.

15.25 The SWE-RL’s bridge

15.25.1 The mechanism

The SWE-RL (arXiv:2502.18449, tier P) is the reinforcement learning on the software engineering’s tasks, with a rule-based reward (the test’s pass/fail). The mechanism:

• The environment: the repository’s state, the tests, the tools (the SWE-agent’s ACI).
• The policy: the model, the harness’s loop.
• The reward: the test’s pass/fail (the verifier), and the intermediate’s signals (the tests’ progress, the lint’s).

The SWE-RL’s role in the curriculum: the bridge from the SFT (Module 15) to the agentic RL (Module 16). The SFT’s data is the demonstration; the RL’s data is the rollout, and the reward is the verifier.

15.25.2 The SFT’s vs the RL’s data

The distinction: the SFT learns from the successful trajectories (the positive examples); the RL learns from the rollouts (the positive and the negative), with the reward’s signal. The SFT’s limitation: it cannot learn from the failures (the negative examples are not in the SFT’s target), and it cannot explore the new solutions. The RL’s advantage: it learns from the failures (the reward’s zero), and it explores.

15.25.3 The bridge’s design

The bridge’s design: the SFT’s model is the RL’s initialisation (the SFT’s warm start), and the RL’s environment is the same harness (the tools, the verifier). The SFT’s dataset (§15.14) is the demonstrations, and the RL’s reward is the verifier (the same, Module 7).

The bridge’s risk: the SFT’s model is overfitted to the teacher’s behaviour, and the RL’s exploration is limited (the policy is far from the teacher’s, and the RL’s signal is sparse). The mitigation: the SFT’s model’s diversity (the teacher’s samples, the multiple solutions), and the RL’s reward’s density (the intermediate’s signals).

15.26 The reward’s design

15.26.1 The rule-based reward

The SWE-RL’s reward is rule-based: the test’s pass (the 1) and the fail (the 0). The rule-based’s advantage: the deterministic’s (the reward is not the model’s, and the calibration is the verifier’s). The rule-based’s disadvantage: the sparse’s (the reward is at the trajectory’s end, and the credit assignment is the long-horizon’s problem, Module 16).

15.26.2 The intermediate’s signals

The intermediate’s signals (the tests’ progress, the lint’s, the type’s) are the dense rewards, and they are the credit’s assignment’s mitigation (Module 16, §16.4’s VPR/RLVP). The signals’ reliability is the verifier’s (Module 7, §7.8), and the signals’ design is the reward’s engineering.

15.26.3 The reward’s hacking

The reward’s hacking is the Module 14, §14.14’s and Module 16’s risk: the policy learns to pass the tests without the fix (the test’s modification, the hard-coded’s output). The mitigation: the immutable verifier (Module 5, §5.13.2), the held-out tests (Module 5, §5.11.2), and the audit (Module 5, §5.13.3).

15.27 The environment’s design

15.27.1 The environment’s requirements

The RL’s environment (Module 17’s subject) must be:

• Hermetic: the repository’s state, the tests, the tools, the no external’s dependency.
• Resettable: the state’s reset to the task’s initial, for the next rollout.
• Instrumented: the trace’s emission (Module 11), the reward’s computation.
• Non-leaky: the tests’ not visible to the model (the hidden tests), the solution’s not in the repository.

15.27.2 The environment’s scale

The environment’s scale is the RL’s cost: the number of the tasks, the rollouts per task, the parallel’s environments. The scale’s arithmetic is the Module 16, §16.8’s, and the environment’s design is the constraint.

15.28 Concept check

Why is the SFT’s dataset the RL’s warm start, and not the RL’s replacement?

Answer. Because the SFT and the RL learn the different things. The SFT learns from the successful trajectories (the imitation), and it cannot learn from the failures (the negative examples) or explore the new solutions. The RL learns from the rollouts (the positive and the negative), with the reward’s signal, and it can explore. The SFT’s warm start is the RL’s initialisation: the SFT’s model is closer to the good behaviour, so the RL’s exploration is more efficient. The SFT’s replacement would be the imitation’s limit: the student mimics the teacher’s behaviour, including the teacher’s weaknesses, and it cannot improve beyond the teacher. The RL is the improvement’s mechanism, and the SFT is the warm start.

LAB 15 — BUILD A FILTERED 5,000-TRAJECTORY SFT SET

15.29.1 Task specification

Build a 5,000-trajectory SFT set from a teacher harness (the harness you built in the previous labs, with a strong model), apply the six filters, and analyse which filtering criteria change the student’s pass^k.

15.29.2 The set’s construction

1. Generate the trajectories: the teacher (the strong model + the harness) on a task set of at least 1,000 tasks, with $n\ge5$ samples per task (the diversity’s floor, §15.7.4).
2. Convert the trajectories: the student’s chat template, the loss mask (the assistant’s spans), the tool-call’s re-serialization (the student’s native format).
3. Filter the trajectories: the six filters (§15.8), in the cost’s order, with the counts at each stage.
4. Build the set: the cumulative filter’s levels, from the no-filter’s set to the all-filter’s set, each with the 5,000’s trajectories (or the maximum’s available).
5. Split: the train/validation/test’s task identifiers, disjoint (the held-out family, §15.19).

15.29.3 The student’s training

6. Train the student on each filter’s level: the same hyperparameters (the learning rate, the epochs, the batch’s size), the same base model.
7. Evaluate each student: the pass^k (the primary, $k=1,2,3$), the parse-failure rate, the cost per correct task, the exploit’s rate (the audit).

15.29.4 The metrics

15.29.5 The hypotheses

State before running:

• H1. The verifier’s filter is the largest pass@1’s improvement, because the outcome’s correctness is the primary’s quality.
• H2. The diversity’s filter is the largest pass^k’s improvement, because the duplicates overfit the student to the specific tasks (Proposition 15.12.2).
• H3. The format’s filter is the largest parse-failure rate’s reduction.
• H4. The length’s and the efficiency’s filters have a small effect on the pass^k, and they bias the set toward the easy tasks (the stratification’s check).
• H5. The filters’ effects interact: the diversity’s filter’s benefit is larger when the verifier’s filter is applied (the verified set’s duplicates are the successful ones, which the student overfits).

15.29.6 The acceptance criteria

15.29.7 What good looks like

A good lab report has five properties.

The pass^k is the primary’s metric. The report’s centerpiece is the pass^k’s curve across the filter’s levels, not the pass@1’s. The pass@1’s improvement is the easy’s finding; the pass^k’s is the reliability’s, and it is the deployment’s question (Module 4, §4.10.3’s delegable horizon).

The filters’ marginal effects are the analysis. The report does not say “the filters improved the student”; it says “the verifier’s filter improved the pass@1 by 8 points, the diversity’s improved the pass^k by 6 points, and the length’s and the efficiency’s had no effect but biased the set toward the easy tasks”. The marginal’s analysis is the finding.

The diversity’s effect is the pass^k’s, and it is explained. The report’s H2’s finding is the duplicates’ penalty, and the explanation is the Proposition 15.12.2’s. A report that finds the diversity’s filter matters, without the mechanism, has the correlation, not the cause.

The bias is checked. The length’s and the efficiency’s filters’ bias (the easy tasks’ over-representation) is the stratification’s check, and the report’s finding is the bias’s magnitude. A report that applies the filters without the bias’s check has the hidden distribution’s shift.

The datasheet is the deliverable’s part. The dataset is not reusable without the datasheet, and the report’s discipline is the datasheet’s completeness.

MODULE 15 CHEAT SHEET

EXERCISES WITH SOLUTIONS

E15.1. A teacher has a 60% per-task success, and the verifier has TPR 0.9, FPR 0.1. Compute the acceptance rate, and the cost per accepted trajectory if $C_g=0.04$ and $C_v=0.01$.

S15.1. The acceptance rate $=0.9\times0.6+0.1\times0.4=0.54+0.04=0.58$. The cost per accepted $=(0.04+0.01)/0.58=0.0862$. The lesson: the verifier’s FPR inflates the acceptance rate (the false accepts), and the accepted set’s quality is the precision (Part 0, §0.5): $\pi_v=0.9\times0.6/0.58=0.54/0.58=0.931$, so 6.9% of the accepted set is wrong.

E15.2. A trajectory has 10 assistant’s turns, each 50 tokens, and 10 observations, each 200 tokens. Compute the mask’s 1-token count and the total’s tokens. What fraction of the sequence is trained?

S15.2. The assistant’s tokens $=10\times50=500$; the observations $=10\times200=2{,}000$; the total $\approx500+2{,}000+$ the prompt’s $\approx2{,}500+$. The trained fraction is $500/2{,}500=20$. The lesson: the mask’s 80% of the tokens are the context, and the training’s signal is the 20% — the observation’s exclusion is the data’s efficiency’s.

E15.3. A set has 5,000 trajectories from 500 tasks (10 per task). What is the diversity’s measure, and what is the effective sample size? If the set is deduplicated to 1 per task, what is the size, and what is the trade-off?

S15.3. The diversity $=500/5{,}000=0.1$ (the task’s coverage is 500 unique tasks, the trajectory’s variety is 10 per task). The effective sample size is the unique tasks’ count, $\approx500$. The deduplication to 1 per task gives 500 trajectories — a much smaller set. The trade-off: the deduplicated set is diverse (500 unique tasks) but small; the full set is large but the effective size is 500. The resolution: the $n$‘s bound (e.g., 2–3 per task), giving $\approx1{,}000$–$1{,}500$ trajectories with the effective size close to the size.

E15.4. A student’s pass@1 is 0.72, and its pass^3 is 0.30. What does the gap suggest, and what would you check?

S15.4. The gap ($0.72$ vs $0.30$) suggests the student’s reliability is low: it succeeds on average, but it fails on the repeated attempts, which means the success is the task’s (the easy tasks) not the student’s (the consistent behaviour). The check: the pass^k’s per-task distribution (the heterogeneity, Module 5, Proposition 0.4.6), the failure’s modes (Module 11, §11.5), and the training’s data (the duplicates’ overfitting, Proposition 15.12.2). The gap is the deployment’s risk (the reliability’s requirement, Module 4, §4.10.3), and the pass@1 alone is misleading.

E15.5. A dataset’s datasheet is missing the privacy’s audit. What are the risks, and what would you do?

S15.5. The risks: (1) the PII’s leak (the training’s set contains the personal data, and the model can reproduce it); (2) the secret’s leak (the API keys, the passwords); (3) the regulation’s violation (the GDPR’s, the sectoral’s). The action: (1) run the PII’s and the secret’s detection on the set; (2) redact or remove the detected data; (3) record the audit’s result in the datasheet; (4) if the audit finds the significant data, the set’s re-collection or the consent’s acquisition. The lesson: the datasheet’s privacy’s field is not optional; the dataset without it is the liability.

E15.6. A distillation’s teacher is a Class 2 model (the acceptable-use, the pass-through), and the student is deployed. What are the licence’s questions?

S15.6. The questions: (1) the teacher’s terms’ pass-through — the student’s deployment must comply with the acceptable-use’s policy; (2) the student’s weights’ licence — the derivative’s licence is the teacher’s, so the student’s distribution’s terms must include the pass-through; (3) the revenue’s or the scale’s conditions — the teacher’s caps apply to the student’s deployment (the derivative inherits the conditions); (4) the attribution — the teacher’s licence’s attribution requirement follows the student. The lesson: the licence’s per-checkpoint’s rule (Module 12, §12.7.1) applies to the derivative, and the student’s deployment is the teacher’s licence’s subject.

E15.7. A set’s trajectories are the teacher’s successful runs, and the teacher’s success is 90%. The verifier’s precision is 0.95. What is the set’s wrong’s fraction, and what is the student’s quality’s bound?

S15.7. The set’s wrong’s fraction is $1-0.95=0.05$ (5% of the accepted trajectories are wrong). The student’s quality’s bound (Proposition 15.9.1) is the set’s quality, so the student’s success on the covered tasks is at most 95% (the set’s correct’s fraction), and the student’s generalisation can be lower. The lesson: the verifier’s precision is the set’s quality’s ceiling, and the 5% wrong’s fraction is the student’s learned’s wrong behaviour’s source.

E15.8. A SFT’s student is worse than the teacher on the deployment’s tasks, and the training’s loss is low. What are the possible causes, and how would you diagnose them?

S15.8. The possible causes: (1) the distribution’s shift — the teacher’s tasks differ from the deployment’s (the set’s distribution, §15.5); (2) the overfitting — the student memorised the set’s tasks, and the generalisation is poor (the pass^k’s gap, Proposition 15.12.2); (3) the capacity’s gap — the student’s model is too small for the behaviour (the knowledge’s transfer, §15.15); (4) the format’s mismatch — the student learned the teacher’s format, not the student’s (the re-serialization, §15.14.2); (5) the mask’s error — the student trained on the observation’s tokens (the hallucination, §15.3.3). The diagnosis: (1) the held-out family’s evaluation; (2) the pass^k’s per-task distribution; (3) the format’s test; (4) the mask’s test; (5) the training’s loss’s curve (the low loss with the poor’s evaluation is the overfitting’s or the shift’s). The lesson: the low training’s loss is not the good’s deployment’s performance, and the diagnosis is the five causes’ check.

EXIT TEST FOR MODULE 15

You are ready for Module 16 when you can, without notes:

1. Define a trajectory and its fields, and state the dataset’s schema.
2. Convert a trajectory to an SFT example, with the mask, the template, and the re-serialization.
3. State the whose-tokens rule and explain the observation’s exclusion.
4. State the five quality dimensions and their measurements.
5. Explain the distribution’s shift and the stratified set.
6. State the rejection sampling and the acceptance rate’s formula.
7. Compute the cost per accepted trajectory.
8. State the six filters, their order, and their interaction.
9. State the verifier’s filter’s precision and the audit.
10. Explain the diversity’s filter and the duplicates’ pass^k’s penalty.
11. Describe the distillation’s pipeline and the re-serialization’s discipline.
12. State what transfers (the format, the behaviour, the knowledge) and the evidence.
13. Distinguish the sequence-level and the logit-level losses and the choice.
14. State the exploit’s amplification and the audit.
15. State the contamination’s, the privacy’s, and the licence’s controls.
16. State the datasheet’s eight sections and the four questions of the undocumented dataset.
17. Describe SWE-RL and the SFT-to-RL bridge.
18. State the reward’s design and the environment’s requirements.

End of Module 15. Say “next” to receive Module 16 in full.