All chapters
  1. Start hereAn agent is a model call in a loop
  2. Chapter 1One call: what a model is from the outside
  3. Chapter 2The loop: where a call becomes an agent
  4. Chapter 3The abstraction ladder: six rungs over one call
  5. Chapter 4The eight jobs of a harness
  6. Chapter 5Context: deciding what the model sees
  7. Chapter 6Tools: what the model may ask for, and where it runs
  8. Chapter 7MCP: one way to plug tools into any harness
  9. Chapter 8RAG: retrieval is context assembly
  10. Chapter 9Control flow and state: stopping, pausing and resuming
  11. Chapter 10Guardrails: identity, approval and the output contract
  12. Chapter 11Reliability and cost: retries, budgets and the failures that will happen
  13. Chapter 12Observability and evals: knowing what the agent did, and whether it was right
  14. Chapter 13Many agents: more loops, and what each one costs
  15. Adopt · 1SDKs: the same loop in Python, TypeScript and Java
  16. Adopt · 2Coding harnesses: a finished harness for software work
  17. Adopt · 3Frameworks and graph runtimes: the loop as a library
  18. Adopt · 4Managed platforms: someone else runs the loop
  19. Adopt · 5Build your own: a whole harness in one file
  20. DecideBuild or adopt: who runs each job?
  21. ReferenceGlossary
  22. AboutAbout this site
Chapter 4

The eight jobs of a harness

Context, tools, control flow, state, identity, output, reliability and observability: the checklist behind every agent product, and the one question to ask of each.

The thirty-line agent from Chapter 2 works. Give it the broker's email, "please quote homeowners for 18 Gulf Shore Blvd, Naples FL, built 1998, 2,400 sq ft", and it asks for the underwriting rules, reads the result and answers with a quote in the high premium band. The first line of its loop, in the function agent of agent_minimal.py, carries a warning written as a comment:

while True:  # no turn limit, no budget, no checkpoint: that is the harness's job
    response = call(messages)

Now suppose the insurer puts those thirty lines in front of its real inbox for a week. People only have to ask ordinary questions. On one email the model kept asking for the same tool: what stopped it? The process restarted during a deploy: where did the open quotes continue? A reply went out to a broker: who approved it? Finance wants the cost per quote. An underwriter wants to know what the agent did on Tuesday's email.

The thirty lines answer none of these, and in none of them did the model do anything wrong. The code around the model is missing.

A harness is easier to reason about as a fixed list of jobs than as a product. Every agent system needs the same eight jobs done. The only thing that changes from one product to the next is who does each job.

1. Eight questions the thirty lines cannot answer#

Chapter 2 ended with six things that can happen to a quote request on a bad day: a loop that does not end, a crash in the middle, a tool request nobody questioned, an answer nobody checked, a call that fails once, and a question a week later.

Those are six. Two more are present, but only in their smallest form. The agent does send a context (everything the model receives in one call): the raw email as the only message, with no instructions and no rules. And it does run a tool: one function, called inside the agent's own process, whatever tool name the model sent.

Put all eight in a row, and each becomes a question that somebody will ask about a real run.

The questionWhat the thirty lines doThe job it becomes
What did the model see when it decided?The raw email, and nothing else. The email is not marked as text from an outside sender.1 Context assembly
What may the model ask for, and where does it run?One tool, run in the agent's own process with no limit on time or access.2 Tools, and where they run
What stops the loop?Only the model. while True ends when the model stops asking for tools.3 Control flow
The process died after step 3. Where does the run continue?Nowhere. The history is a Python list in memory.4 State, memory, durability
Who is this run acting for, and who must approve a reply?Nobody is identified, and nothing asks.5 Identity, permissions, approval
Can the next program rely on the shape of the answer?The answer is whatever text the model returned.6 Output contract
The API answered with an error. What happened, and what did the run cost?The exception ends the run. No token is counted.7 Reliability and cost
What did the agent do on Tuesday's email?No record exists.8 Observability and evals

These are the eight jobs of a harness. Their names, numbers and order are fixed on this site, because every later chapter and product page refers to them. None of them makes the model smarter. Each is ordinary software around a model call.

2. One call in the middle, eight jobs around it#

The picture to keep is small. In the middle is the one thing the model does: it receives a context and returns text or a tool request. It keeps nothing and runs nothing. Around that call sit the eight jobs, all of them code.

Diagram: One model call in the middle · eight jobs around it

The center is the only thing the model does. Every box around it is ordinary code, and together they are the harness. Read clockwise from the left: jobs 1 and 2, then 3 and 4 at the top, 5 and 6 on the right, 7 and 8 at the bottom. Click any box.

  1. The model call, in a loop: One request to the model, repeated. The model receives a context and returns text or a tool request. It keeps nothing between calls and runs nothing itself.
  2. In: the whole context: Everything the model receives in one call: instructions, tool descriptions, retrieved documents and the history so far.
  3. Out: text or a tool request: The model returns either a final answer or a request to run a tool. The harness reads the stop reason to decide what happens next.
  4. What the model sees
    1. 1 · Context assembly: Decides what the model sees before every call: instructions, rules, retrieved documents, history.
    2. 2 · Tools, and where they run: Describes the tools to the model and runs each tool call inside a boundary. The model asks, the harness decides and calls, the tool does.
  5. Keeping the run going, and ending it
    1. 3 · Control flow: Runs the loop and ends it for a named reason: done, turn limit, budget, waiting for a person.
    2. 4 · State, memory, durability: Saves the run after every step, so it can continue after a crash or after a person has decided.
  6. Limits on what leaves
    1. 5 · Identity, permissions, approval: Knows who the run acts for. Decides which data and tools the run may use, and when a person must approve.
    2. 6 · Output contract: Checks the final answer against a fixed shape before any other program uses it.
  7. When it fails, and what happened
    1. 7 · Reliability and cost: Handles the failures that will happen and keeps a limit on spending.
    2. 8 · Observability and evals: Records each run as a trace, and tests outputs automatically so a change can be judged.

Read it clockwise from the left, jobs 1 to 8. The checklist follows the same order. Each job gets one sentence on what it is, the question it answers for the Naples quote, and the functions that do it in the miniature: examples/quote-email/harness.py, about 450 lines of standard-library Python with every job as a named function.

Job 1 · Context assembly#

Context assembly decides what enters the context, and in what order, before every call. The question in the quote example: which instructions, which underwriting rules and how much history does the model see when it decides about the Naples house? assemble_context builds the request body, retrieve_rules picks the rule documents, and wrap_email fences the email so it reads as data from an outside sender. Depth: Context, and RAG for retrieval.

Job 2 · Tools, and where they run#

This job describes the tools to the model and runs each tool call inside a boundary. The model asks, the harness decides and calls, the tool does. The question here: may this run call lookup_property, and what happens when that call hangs? TOOL_SCHEMAS is what the model sees, and run_tool_guarded checks an allow-list and a wait limit before it calls anything. Depth: Tools, and MCP for tools in a separate program.

Job 3 · Control flow#

Control flow runs the loop and ends it for a named reason. The question: when does work on this email end, and with which status? run is the loop, and model_step and tool_step are its two kinds of step. Every exit has a name, such as done, stopped:max_turns or waiting_approval. Depth: Control flow and state.

Job 4 · State, memory, durability#

The model remembers nothing, so this job saves the run after every step and loads it again later. The question: if the process dies after the property lookup, or an underwriter approves two days later, where does the run continue? start_run creates the state, checkpoint writes it to SQLite after each step, and load_checkpoint reads the newest one. Depth: Control flow and state.

Job 5 · Identity, permissions, approval#

This job knows who the run acts for. From that it decides which data and tools the run may use, and when a person must say yes. The question: may this caller email a broker, and what happens when the inbound email contains text that reads like an order? Caller carries the role, tool_schemas_for and may_read narrow what the model is offered, authorize is the gate before every tool call, and screen_untrusted and enforce_policy handle a suspicious email. Depth: Guardrails.

Job 6 · Output contract#

An output contract is the shape the final answer must have before any other program uses it. The question: is the answer exactly decision with one of three values, plus missing_fields and reasons, or is it a friendly paragraph? validate_decision checks the answer, and repair_prompt asks the model once to correct it. Depth: Guardrails.

Job 7 · Reliability and cost#

This job handles the failures that will happen and keeps a limit on spending. The question: what happens when the API is overloaded, how many tokens may one quote use, and can a resumed run send the same email twice? call_with_retry and pick_model retry and fall back to a second model, over_budget stops a run at its token budget, and send_once makes sure a reply is recorded once. Depth: Reliability and cost.

Job 8 · Observability and evals#

A trace is the record of one run. An eval is an automated test of outputs. The question: what did the agent do on this email, and does the harness still decide five known emails correctly after an edit? trace writes one line per model response and per tool call, and run_case and grade in evals.py score the five golden cases. Depth: Observability and evals.

Later chapters quote these functions. Build your own reads the whole file in order.

Three boundaries between the jobsWhere one job ends and its neighbour begins

Some pieces could sit under two jobs. This site puts each in one place.

PairWhere the line is
State (4) and reliability (7)State owns checkpoints and resume. Reliability owns the retry policy and sending each side effect once.
Tools (2) and identity (5)Tools owns the mechanics and the boundary a tool runs in. Identity owns who may run what, and approval.
Output (6) and reliability (7)Output owns checking and repairing what the model wrote. Reliability owns retries of the call itself and of tools.

All eight in one run#

The next command runs the miniature on the Naples email against a scripted fake model, with no network and no key. It shows the jobs as visible events. A successful run proves that the harness code paths work in the order described. It proves nothing about a real model, because the replies are scripted.

cd examples/quote-email
/opt/homebrew/bin/python3 harness.py --fake --verbose
{"run_id": "run-da7f0eba", "step": 1, "turn": 1, "kind": "model", "model": "claude-opus-5-5", "stop_reason": "tool_use", "input_tokens": 61, "output_tokens": 31, "cache_read": 0, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 2, "turn": 1, "kind": "tool", "tool": "verify_address", "verdict": "allow", "is_error": false, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 3, "turn": 2, "kind": "model", "model": "claude-opus-5-5", "stop_reason": "tool_use", "input_tokens": 145, "output_tokens": 31, "cache_read": 526, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 4, "turn": 2, "kind": "tool", "tool": "lookup_property", "verdict": "allow", "is_error": false, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 5, "turn": 3, "kind": "model", "model": "claude-opus-5-5", "stop_reason": "tool_use", "input_tokens": 227, "output_tokens": 42, "cache_read": 526, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 6, "turn": 3, "kind": "tool", "tool": "check_underwriting_rules", "verdict": "allow", "is_error": false, "latency_ms": 0}
{"run_id": "run-da7f0eba", "step": 7, "turn": 4, "kind": "model", "model": "claude-opus-5-5", "stop_reason": "tool_use", "input_tokens": 328, "output_tokens": 51, "cache_read": 526, "latency_ms": 0}
send_reply is waiting for a human. Continue with: --resume run-da7f0eba --approve
{
  "run_id": "run-da7f0eba",
  "status": "waiting_approval",
  "decision": null,
  "reply": "waiting_approval",
  "turn": 4,
  "tokens": 3020
}

Almost every field belongs to a job. The lines themselves are the trace (job 8). Each tool line carries a verdict, the answer of the permission gate (job 5), before the guarded call (job 2). The step and turn counters and the status are control flow (job 3). The run stopped because the model asked for send_reply and no person had approved it (job 5). It can continue days later by its run_id, because every step was saved (job 4). tokens is the count the budget is checked against (job 7). cache_read is job 1: the fixed front of the context was built to be reused (the fake model simulates the number). decision is still null: the output contract (job 6) is checked when the model gives its final answer, after the approval.

3. Not every step needs a loop#

Before filling in the eight jobs, one design decision comes first, because it changes how much of each job you need: where should a loop be used at all?

In a loop, the model picks the next step. That is valuable when nobody can say in advance what the next step will be. It also has a price. Every extra turn sends the whole context again, so cost grows with each turn. The path can differ between two runs on the same input, so testing is harder. And the loop needs stop conditions, because otherwise the model alone decides when it is finished.

When the next step is known, there is a cheaper form. Code calls the steps in a fixed order. That is a workflow: code fixes the order of the steps, and a model, if one is used, works inside a step. A step can be plain code, a single model call with one structured answer and no tools, or an agent loop.

Look at the miniature with that in mind. Its system prompt says: "Verify the address, look up the property, then check the underwriting rules." The order is known, so code could call those three tools in sequence. The miniature uses a loop anyway, because it exists to show the loop and its jobs in one file.

The full build of the same use case decides stage by stage. It is described on this site, not shipped with it, and its numbers come from that build. It has seven stages. Code fixes their order, and a model never chooses the next stage.

Diagram: The full build · seven stages, and who acts in each

Columns are the seven stages, left to right. Code fixes that order. Each lane is a kind of actor. Only two boxes sit in the agent loop lane: those are the only places where a model picks the next step.

  1. Code
    1. Clean the request, verify the address: Code cleans the request, builds a key so a request sent twice updates the same quote, and verifies the address.
    2. County record, value estimate: One fixed lookup fetches the county record and an automated valuation.
    3. Re-check the tier: Code checks the tier the model returned against the same criteria.
    4. 29 rules and the price: Code runs the rule playbook and the price first. If every point is clear, no model is called.
    5. Send a question from a template: When information is missing, a question goes out by email from a template, with no model.
  2. One model call
    1. Choose the tier: One request, one structured answer: the tier. No tools, so there is no next step to choose.
    2. Read the reply: One model call reads the broker's free-text reply into fields. The quote is then classified again.
  3. Agent loop
    1. Fill the form: The model plans which data services this tier needs, asks for the lookups, then asks to save values and flag conflicts. The harness calls each tool.
    2. Resolve flagged points: Only for points the rules flagged: the model reads the facts, asks for a search of past decisions, and recommends.
  4. A person
    1. Underwriter decides: A person approves, asks for information or declines. The premium tier needs a senior underwriter.

Four steps use a model: two single calls and two agent loops. Everything else is code or a person.

StageWho actsWhy this form
1 IntakeCodeThe same steps for every request.
2 ValuationCodeOne fixed lookup.
3 ClassifyOne model call (small model), then code re-checksOne question, one structured answer. There is no next step to choose.
4 PrefillAgent loop (small model, at most 6 turns)Which data services are needed depends on the tier, and conflicts appear only as results arrive.
5 Underwriting resolveCode first: 29 rules and the price. Then an agent loop (larger model, at most 10 turns), only for flagged pointsThe rules are known in advance. What to look up for an open point is not.
6 DecisionA personThe model recommends; a person decides. The premium tier needs a senior underwriter.
7 Email and learningCode, plus one model call to read a replyThe question goes out from a template. The broker's reply is free text.

Stage 5 shows the rule most clearly. Code runs first. If the rules clear every point, no model is called at all. The loop starts only for the flagged points, where the next useful lookup depends on what the last one returned.

Use a loop only where the next step cannot be known in advance.

A workflow has its own limit. It handles only the paths its author foresaw, and a new kind of case needs a code change. A loop handles variety, at the price of a cost and a path that are harder to predict. That is why the full build is a workflow on the outside with two bounded loops inside.

The decision also changes the checklist. The outer workflow is job 3 done by plain code. Each loop inside it gets its own turn limit. A single model call needs no loop control, and still needs its context assembled (job 1), its answer checked (job 6), its failures handled (job 7) and its trace written (job 8). The eight jobs apply to every model step. A workflow makes several of them smaller.

4. The question for every product: who runs this job?#

Chapter 3 described six rungs, from raw HTTP up to a managed harness, and said that each rung hides some work. The eight jobs make "some work" exact. A product is a set of answers to one question, asked eight times: who runs this job?

This site uses five answers. YOU BUILD: the job is your code, as in the miniature. BUILT IN: the product ships the job, and you still operate it. INTEGRATES: the product hands the job to a named partner product. MANAGED: the vendor runs the job as a service. NOT ITS JOB: the product does not aim at that job, which is a fact about its scope and not a fault.

One job is enough to see all five: job 4, saving the run so it can continue.

AnswerExample for job 4What the documentation says
YOU BUILDThe miniaturecheckpoint and load_checkpoint write and read a SQLite row after every step.
BUILT INLangGraphCheckpointers persist a thread's graph state as checkpoints.
INTEGRATESPydantic AIDurable execution runs on other engines. Its overview names nine, among them Temporal, DBOS and Prefect.
MANAGEDClaude Managed Agents (beta)The service stores the conversation history server-side and keeps the event history.
NOT ITS JOBLangfuseIt describes itself as a platform for tracing, prompt management and evaluation. It records runs; this site reads its scope as not including running or resuming them.

The product rows were read on each vendor's own documentation on 2026-11-16. The table is not a ranking. Each answer has a cost: your engineering time, a library to operate, a second product to run, or a vendor who decides how the job is done.

A sixth entry is deliberate: "the docs do not say". When a vendor's documentation is silent about a job, this site writes that and does not guess.

Each job chapter ends with that job across products, and the full table of products and jobs is on Build or adopt.

5. A checklist first, the depth next#

You can now take any agent, from thirty lines to a vendor's platform, and ask eight questions of it. You can also ask, step by step, whether a loop is needed or whether code already knows the order. What you cannot do yet is any of the jobs well: this page gave each one a sentence.

The first job is the one the others depend on. The model sees nothing except the context, and every step of a loop sends the whole context again. What enters it, in what order, and what that costs on every turn is the first design problem. Chapter 5 starts there.

An agent product is not a new kind of thing. It is one answer, eight times, to "who runs this job?", and the jobs do not go away when someone else runs them.