All chapters
  1. Start hereAn agent is a model call in a loop
  2. Chapter 1One call: what a model is from the outside
  3. Chapter 2The loop: where a call becomes an agent
  4. Chapter 3The abstraction ladder: six rungs over one call
  5. Chapter 4The eight jobs of a harness
  6. Chapter 5Context: deciding what the model sees
  7. Chapter 6Tools: what the model may ask for, and where it runs
  8. Chapter 7MCP: one way to plug tools into any harness
  9. Chapter 8RAG: retrieval is context assembly
  10. Chapter 9Control flow and state: stopping, pausing and resuming
  11. Chapter 10Guardrails: identity, approval and the output contract
  12. Chapter 11Reliability and cost: retries, budgets and the failures that will happen
  13. Chapter 12Observability and evals: knowing what the agent did, and whether it was right
  14. Chapter 13Many agents: more loops, and what each one costs
  15. Adopt · 1SDKs: the same loop in Python, TypeScript and Java
  16. Adopt · 2Coding harnesses: a finished harness for software work
  17. Adopt · 3Frameworks and graph runtimes: the loop as a library
  18. Adopt · 4Managed platforms: someone else runs the loop
  19. Adopt · 5Build your own: a whole harness in one file
  20. DecideBuild or adopt: who runs each job?
  21. ReferenceGlossary
  22. AboutAbout this site
Chapter 1

One call: what a model is from the outside

A hosted model is an HTTP endpoint that keeps nothing between calls. What goes in, what comes back, and the control signals that make everything else possible.

The insurer's team sends the broker's email to a model. One HTTP request goes out, and a reply comes back in a few seconds. The reply says: "I'll verify the address first."

Then nothing happens. No address is verified. No second message arrives. The connection is closed, and the program is holding a sentence that promises work nobody is doing.

So a developer sends a second request to find out: "Is the address valid?" The model answers politely that it has not been given an address. It has no trace of the email it read ten seconds ago.

Both surprises have the same cause, and every part of an agent system exists because of it. This chapter looks at one call, slowly: what goes in, what comes back, and what is gone the moment it ends.

A hosted model is a function over HTTP. It reads one request, writes one response, and keeps nothing. Whatever must last longer than one call has to live in software around the call.

1. What the team is talking to#

The use case is the one from the start page: a broker asks an insurer for a homeowners quote by email, and the system must answer with a priced quote, a question, or a referral to a person. The email is one line: "please quote homeowners for 18 Gulf Shore Blvd, Naples FL, built 1998, 2,400 sq ft".

The team wants a language model to do the first pass. On this site, model means the language model alone: a program that reads text and writes text. The team does not run it. A vendor runs it on its own servers and offers it as an HTTP endpoint (a URL that accepts requests). The program posts JSON to that URL and receives JSON back.

That is all the model is from the outside. There is no session to open, no connection to keep, no object that represents "the conversation". For Claude models the endpoint is called the Messages API, and its reference describes it as usable for "single queries or stateless multi-turn conversations". The word that matters is stateless. Section 4 makes it concrete. First, the request.

2. The request: everything the model will know#

Whatever the model should take into account must be inside the request body. That is the rule that shapes the request. The model cannot look anything up, and it has no earlier call to draw on. If a fact is not in the JSON, it does not exist for this call.

The script below is examples/quote-email/curl_example.sh, copied whole. It shows the smallest useful request for the quote example: instructions, one tool the model may ask for, and the broker's email. It was not run while this site was built. It follows the request shape in the API reference, which was read on 2026-11-16. A successful run would prove that the key, the headers and the body are accepted. It would not prove the answer is right.

#!/bin/sh
# One call to the Messages API. Content in; content and control signals out.
# The key comes from the environment. Never paste a real key into a file.
# The documented shape of the reply is in sample_response.json (illustrative).
curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2026-10-21" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 16000,
    "system": "You triage inbound homeowners quote emails for an insurance carrier.",
    "tools": [{
      "name": "verify_address",
      "description": "Check that a street address exists.",
      "input_schema": {
        "type": "object",
        "properties": {"address": {"type": "string", "description": "Street, city and state"}},
        "required": ["address"]
      }
    }],
    "messages": [
      {"role": "user", "content": "Please quote homeowners for 18 Gulf Shore Blvd, Naples FL, built 1998, 2,400 sq ft."}
    ]
  }'

Read it from the top. It is a POST to /v1/messages with three headers. x-api-key says who is paying. anthropic-version pins the API version, so that a later change to the API does not change what this program receives. content-type says the body is JSON. The key is read from an environment variable and never written into the file.

The body has five fields, and each one answers a question the model cannot answer for itself.

Which model, and how long may it write? model names the model. max_tokens is a ceiling on the length of the reply. Length is counted in tokens (pieces of words; section 5 explains them). The ceiling is a safety limit, not a target: the reference notes that a model may stop before reaching it.

What is its job? system carries the system prompt: standing instructions that apply to the whole call. Here it is one sentence that says the model triages quote emails for an insurer. It sits outside the conversation because it is not something the broker said. It is what the insurer's program says about how to treat everything else.

What may it ask for? tools is a list of tool definitions. Each has a name, a description and an input_schema: a JSON Schema that says which arguments the tool takes. This definition does not give the model the ability to verify an address. It only tells the model that such a function exists on the caller's side, and what a well-formed request for it looks like. Tools is the home of schemas and of where tools run.

What was said so far? messages is the conversation as a list. Each entry has a role, user or assistant, and a content. This first request has one entry: the broker's email, in the user role. The role name is historical. It means "the caller's side", not "a person": the insurer's program wrote this entry.

The table collects the five fields. The descriptions in the third column follow the API reference.

FieldIn this requestWhat it is for
modelclaude-opus-5-5Which model answers
max_tokens16000The most tokens the model may generate before it is stopped
systemOne sentence about triageThe system prompt: context and instructions for the whole call
toolsOne definition, verify_addressTools the model may ask the caller to run
messagesOne user entry, the emailThe prior turns of the conversation, in order

One thing is missing from the table, and its absence is the point of this chapter. There is no conversation id, no session token and no "continue from last time" field in this request. Nothing in it refers to any earlier call.

The three headers, and what the documentation says about eachRead on 2026-11-16; sources in the fact log
HeaderIn the exampleWhat the documentation says
x-api-key$ANTHROPIC_API_KEYThe API key. The API overview also lists Authorization: Bearer <token> and calls x-api-key a legacy fallback that is still supported. The curl examples in the documentation use x-api-key.
anthropic-version2026-10-21The API version. The versioning page lists 2026-10-21, 2023-06-01 and 2023-01-01, and recommends the latest.
content-typeapplication/jsonRequired. The body is JSON.

A version is a contract about the request and response shape. The versioning page says that within a version, existing input and output parameters are preserved. A new version may remove something. Version 2026-10-21 removes the request parameters temperature, top_p and top_k: a request that sends one of them returns a 400 error.

Every response also carries a request-id header. It is the value to log, because it is what the vendor's support asks for.

3. The response: content, and signals for the caller#

The reply is one JSON object. It holds two different kinds of thing, and telling them apart is most of the skill of reading it. Some of it is content: what the model wrote. The rest is control signals: fields that tell the calling program what state the reply is in and what to do next.

The file below is examples/quote-email/sample_response.json. It is illustrative. It was written by hand from the API documentation to show the documented shape of a reply to the request above. It was not captured from a real call: the ids, the text and the token counts are invented. A real reply can carry more blocks and more fields, for example a stop_details field next to stop_reason.

{
  "_note": "ILLUSTRATIVE. Hand-written from the Messages API documentation to show the response shape for curl_example.sh. Not captured from a real call: ids, text and token counts are invented, and a real reply can carry extra blocks (for example a thinking block) and extra usage fields.",
  "id": "msg_01EXAMPLEEXAMPLEEXAMPLE",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-5-5",
  "content": [
    {
      "type": "text",
      "text": "I'll verify the address first."
    },
    {
      "type": "tool_use",
      "id": "toolu_01EXAMPLEEXAMPLEEXAMPLE",
      "name": "verify_address",
      "input": {
        "address": "18 Gulf Shore Blvd, Naples FL"
      }
    }
  ],
  "stop_reason": "tool_use",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 412,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 58
  }
}

The content is a list of blocks, not a string#

A first-time reader expects content to be text. It is a list, because a model can return more than one kind of thing in one reply. Each item is a content block with a type.

This reply has two blocks. The first has type text: the sentence from the opening of this chapter. The second has type tool_use, and it is not prose. It holds the name of a tool from the request, an input object that follows that tool's schema, and an id that identifies this one request for a tool.

A program that prints only the text blocks sees "I'll verify the address first." and nothing else. That is the first surprise from the opening. The model did say what it wanted. It said it in a block the program was not reading.

The stop reason says why the model stopped writing#

Every reply ends for a reason, and stop_reason names it. Here the value is tool_use: the model stopped because it is waiting for the result of a tool. The reply is not an answer to the broker. It is a request to the caller.

Had the model simply finished, the value would be end_turn. Had it run into the ceiling from the request, the value would be max_tokens, and the text would be cut off mid-thought. These three cases need three different reactions from the program, and the text alone does not say which case it is. A reply that ends in a full sentence can still be a truncated one.

That is why this site calls stop_reason a control signal. It is written for code, not for a reader. The first line of any program that handles a reply should branch on it.

stop_reasonWhat happenedWhat the caller does
end_turnThe model completed its responseUse the content
tool_useThe model asks for a toolRun the tool and send the result in a new call. The loop is about exactly this.
max_tokensThe reply reached the max_tokens limitTreat the content as cut off
All seven documented stop reasonsFrom the stop-reasons page of the documentation, read on 2026-11-16
stop_reasonThe documentation's descriptionWhat it means for the caller
end_turn"Claude completed its response naturally."The normal end
max_tokens"The response reached your max_tokens limit."The content is truncated
stop_sequence"Claude emitted one of your stop_sequences."The caller asked for this stop; stop_sequence holds the text that matched
tool_use"Claude is calling a tool."Run the tool and return the result
pause_turn"A server-tool loop reached its iteration limit."Send the assistant content back to continue. This applies to tools the vendor runs.
refusal"Claude declined to respond."Do not treat the content as an answer
model_context_window_exceeded"The response filled the model's context window."Treat the response as truncated

The documentation's phrase is that the model "is calling a tool". This site words it differently on purpose: the model asks, the harness decides and calls, the tool does. For a tool defined in the request, as verify_address is, nothing runs until the caller runs it.

The usage block is the meter#

usage counts what this call consumed, in tokens. input_tokens is what the model read. output_tokens is what it wrote. The two fields with cache in their names are also input: the documentation says the input count is split across the three fields. They are zero here, and Context explains when they are not.

The numbers in the sample are invented. What matters is the shape of the bill. The caller pays for input on every call, and input is everything in the request: the documentation counts the system prompt, every message and the tool definitions. So the one-line email is not a one-line input.

The diagram puts the two halves side by side.

Diagram: One call · what goes in, what comes back

Read left to right. The left column is the request the caller builds. The middle is one run of the model. The right column is the reply. Two of the three boxes on the right are signals for a program, not text for a person. Click any box.

  1. Request · what the caller sends
    1. system: The system prompt: standing instructions for the whole call.
    2. tools: Definitions of tools the model may ask the caller to run: name, description, input schema.
    3. messages: The whole conversation so far, as a list of user and assistant entries.
    4. model + max_tokens: Which model answers, and the most tokens it may generate.
  2. The model · one inference step
    1. HTTPS POST /v1/messages: One request to the Messages API. No session is opened and none is kept.
    2. Reads the request, writes a reply: The model reads the context in this request and generates a reply. It can read nothing else.
    3. Keeps nothing: When the reply is sent, nothing about this call remains on the model's side.
  3. Response · what comes back
    1. content: A list of blocks: text the model wrote, and tool_use blocks that ask for a tool.
    2. stop_reason: Why the model stopped writing. The caller branches on this before it reads the text.
    3. usage: Tokens read and tokens written in this call. The meter for cost.

Read the right-hand column again. Two of its three boxes are not for a person to read. A model reply is a message to a program, and the program has to act on it.

4. Send the same request twice#

Now the second surprise. Suppose the program sends the request from section 2 twice, one second apart.

The endpoint treats them as two unrelated requests. Each one is read from the first byte, answered and forgotten. The second reply does not say "as I mentioned". It asks for verify_address again, as if for the first time, because for the model it is the first time.

The two replies may also differ in wording. The vendor's glossary says that identical inputs might produce different outputs across API calls. So the caller cannot treat a repeated request as a way to get the same reply back.

This is what stateless means. Nothing about a call survives on the model's side: not the email, not the reply, not the fact that a call happened. What a vendor may log or store for billing and safety is a separate matter, and the model does not read from it.

Now the developer's follow-up from the opening makes sense. It is shown here as a mistake. This body is illustrative and was not sent:

{
  "model": "claude-opus-5-5",
  "max_tokens": 16000,
  "messages": [
    {"role": "user", "content": "Is the address valid?"}
  ]
}

It is a valid request, and that is the trap. The API accepts it. But the model receives nine words and nothing else. No email, no address, no system prompt, no tool. The mistake is the assumption that the endpoint is holding the conversation.

The rule that fixes it: the caller keeps the conversation and sends all of it, every time. A correct second request repeats the system prompt and the tools, repeats the email as the first message, adds the model's own reply as an assistant message, and only then adds what is new. The model reads its own earlier words as input. That is how it "remembers" them.

Diagram: Two calls · the caller carries the conversation

Time runs down the page. The endpoint answers each request from what that request contains. Between two calls, only the caller holds anything. The second request is larger because it contains the first.

  1. The insurer's program to Model endpoint: Call 1: system, tools, the email. Everything the model will know is in this request.
  2. Model endpoint to The insurer's program: Reply 1: a tool request. The stop reason is tool_use. The call is over.
  3. Model endpoint: Nothing is kept. The endpoint holds no conversation. The email and the reply are gone on its side.
  4. The insurer's program: Appends reply 1 to its list. The only copy of the conversation is in the caller's memory.
  5. The insurer's program to Model endpoint: Call 2: system, tools, the email, reply 1, what is new. The caller sends the whole history again. The model reads its own earlier reply as input.
  6. Model endpoint to The insurer's program: Reply 2. Answered from the contents of call 2 alone. Input tokens are higher than in call 1.

The miniature shows this with code that runs. The function below is call_model from examples/quote-email/agent_minimal.py. It is the same request as the curl script, written with the Python standard library. Notice that it takes the whole messages list as its argument. It has nothing else to send.

def call_model(messages: list[dict]) -> dict:
    """One HTTPS POST. The model keeps nothing, so every call carries the full history."""
    body = {"model": "claude-opus-5-5", "max_tokens": 16000, "tools": TOOLS, "messages": messages}
    request = urllib.request.Request("https://api.anthropic.com/v1/messages", json.dumps(body).encode(), {
        "x-api-key": os.environ["ANTHROPIC_API_KEY"],
        "anthropic-version": "2026-10-21",
        "content-type": "application/json"})
    with urllib.request.urlopen(request, timeout=120) as reply:
        return json.load(reply)

The run below uses the scripted fake model, so it needs no key and no network. --verbose prints each model response. The script makes two calls, because it contains the loop that the next chapter reads. For this chapter, look at one number only: input_tokens in each response.

cd examples/quote-email
/opt/homebrew/bin/python3 agent_minimal.py --fake --verbose
{"id": "msg_fake_01", "type": "message", "role": "assistant", "model": "claude-opus-5-5", "content": [{"type": "tool_use", "id": "toolu_fake_01", "name": "check_underwriting_rules", "input": {"address": "18 Gulf Shore Blvd, Naples FL", "year_built": 1998, "sqft": 2400}}], "stop_reason": "tool_use", "stop_sequence": null, "usage": {"input_tokens": 109, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "output_tokens": 42}}
{"id": "msg_fake_02", "type": "message", "role": "assistant", "model": "claude-opus-5-5", "content": [{"type": "text", "text": "{\"decision\": \"quote\", \"missing_fields\": [], \"reasons\": [\"eligible; flood zone AE\"], \"premium_band\": \"high\"}"}], "stop_reason": "end_turn", "stop_sequence": null, "usage": {"input_tokens": 210, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "output_tokens": 37}}
{"decision": "quote", "missing_fields": [], "reasons": ["eligible; flood zone AE"], "premium_band": "high"}

The first call reports 109 input tokens. The second reports 210. Nothing new arrived from the broker between the two. The second request is larger because it contains the first request again, plus everything that happened since.

Be clear about what this run proves. It proves that the caller in this file resends the history: the request body grew. It does not measure real token counts. The fake model has no tokenizer and estimates a token as four characters of JSON. This miniature also has a different single tool, check_underwriting_rules, from the curl script.

5. Resending has a size, and the size has a limit#

If every call carries the whole conversation, two questions follow. How big can a request get? And what does the size cost?

Both are measured in tokens. A model does not read characters or words. Its input is first cut into small units, and the vendor's glossary says a token can correspond to a word, a part of a word, a character or even a byte. A short common word is often one token. A street address or a policy number breaks into several. Everything about a call is counted in this unit: the ceiling in max_tokens, the numbers in usage, the bill, and the limit below.

The limit is the context window. The context is everything sent to the model in one call. The context window is the most the model can hold at once: the documentation defines it as all the text the model can reference when generating a response, including the response itself. So input and output share one budget.

For the model in the example, claude-opus-5-5, the models overview lists a context window of 1M tokens and a maximum output of 128K tokens, as read on 2026-11-16. One email is nowhere near that. A conversation that grows with every call gets there in the end, and two things happen at the edge:

A caller does not have to guess the size. A second endpoint, POST /v1/messages/count_tokens, takes the same inputs as a message request and returns input_tokens without running the model. The documentation says it is free to use, subject to rate limits, and that the count is an estimate.

The limit is rarely the first problem. The cost is. A conversation of ten calls does not cost ten times the first call. Each call resends everything before it, so the tenth call pays again for the nine before. This is the reason a later chapter treats what enters the context as the main design decision. The remedies have names: prompt caching, which lowers the price of a repeated prefix, and compaction, which replaces old turns with a summary. Their home is Context. Here it is enough to know why they exist: the model keeps nothing, so the caller sends everything, so everything is paid for again.

6. What one call cannot do#

It is worth collecting what the team has learned, because each item is a gap that some later piece of software has to fill. None of them is a defect of the model. They follow from the shape of the call.

It cannot remember. Section 4 showed it. Something must keep the conversation between calls, and keep it through a crash if the work matters.

It cannot act. The tool_use block is a request. The address is not verified until some code on the caller's side runs verify_address. The same holds for every tool the caller defines: the model asks, the harness decides and calls, the tool does.

It cannot continue. After a tool_use reply the model is not waiting. The call is over. If the result of the tool should reach the model, somebody has to make a new call that contains it.

It cannot be trusted to be complete or well-formed. A reply can be cut off at max_tokens. It can be a refusal. Its text can claim to be a decision without being valid JSON. The caller has to check.

It can fail outright. The endpoint is a network service. The errors page of the documentation lists, among others, status 429 for a rate limit and 529 when the API is temporarily overloaded.

What one call lacksWhyWhere this site deals with it
Memory between callsThe endpoint is statelessContext, and Control flow and state
The ability to actA tool request is only a messageThe loop, then Tools
A next stepA call ends when the reply is sentThe loop
A checked outputThe reply is whatever the model wroteGuardrails
A guarantee of an answerRate limits, overload, truncationReliability and cost

Every row of this table is work for the software around the call. That software is the harness, and the rest of the site is about it.

7. Who keeps the conversation?#

The model does not. So something else does, and there are only a few places it can be. This is the first time the site's standing question appears: who runs this job, you, a library, or a vendor?

Your own code can keep it. That is what agent_minimal.py does: a Python list named messages, appended to after every call and sent whole with the next one. It is the plainest arrangement and nothing is hidden. It also means that if the process dies, the list dies with it. The larger miniature, harness.py, writes a checkpoint to SQLite after every step for that reason. Control flow and state covers that.

A library in your process can keep it. A provider SDK wraps the HTTP call and leaves the list to you. An agent SDK or a framework holds the list for you and resends it. The request on the wire is the same one as in section 2. The ladder sorts these layers.

A vendor can keep it. Some APIs store the conversation on the server, and the next request refers to it by an id. The OpenAI Responses API offers this as an option, with previous_response_id and durable conversation objects. That does not make the model remember. A different program holds the state, and each inference step (one run of the model) still receives the whole context.

Where the conversation livesExampleWho runs the job
In the caller's codeMessages API with a messages list, as in agent_minimal.pyYOU BUILD
In a library inside the caller's processAn agent SDK or framework over the same APIBUILT IN to the library; the process and its storage are still yours
On the vendor's serversOpenAI Responses API with previous_response_id; the baseline fact check records that responses are stored for 30 days by defaultMANAGED, as an option

The table is not a ranking. It says where to look when a conversation is lost, grows too large or must be deleted: at whoever holds the list.

One more note on the word HTTP. Hosted models are commonly reached this way, and this site uses that case throughout. A model that runs on your own machine is a function call inside your process. The shape does not change: content in, content and signals out, nothing kept.

What is, and is not, run on this pageOne fake-model run; no live API call
Shown on this pageStatus
curl_example.shFollows the documented request shape. Not run against the real endpoint. Its syntax is checked with bash -n.
sample_response.jsonIllustrative. Hand-written from the documentation; ids, text and token counts are invented.
The "Is the address valid?" requestIllustrative. A labelled mistake; not sent.
agent_minimal.py --fake --verboseRun on 2026-11-16. The output is pasted unchanged. The model is the scripted fake; its token counts are estimates.
Field names, headers, the endpoint, the model id, stop reasons, limitsRead in the vendor's documentation on 2026-11-16 and listed in the fact log of this site.

The behavior described in section 4, two identical requests answered independently, follows from the documented design of the API. It was not observed in a live call for this page.

8. The reply asked for something, and nobody is there#

Go back to the reply in section 3 with what you now know. The team can read it completely: a text block, a tool_use block that names verify_address and gives the address, a stop reason that says the model is waiting, and a meter.

And still nothing in one call can answer it. The model has stopped. The endpoint has forgotten. The address is unverified, and the only thing in the world that knows a tool was asked for is the program holding this JSON. If that program runs the tool, puts the result into the list and calls again, the work continues. If it does not, the sentence "I'll verify the address first" stays a promise.

A single call is a pure function: the same kind of thing every time, with no past and no hands. Everything that looks like an agent comes from what the caller does between calls. The smallest version of that is a loop, and the next chapter builds it in about thirty lines.