The insurer's team sends the broker's email to a model. One HTTP request goes out, and a reply comes back in a few seconds. The reply says: "I'll verify the address first."
Then nothing happens. No address is verified. No second message arrives. The connection is closed, and the program is holding a sentence that promises work nobody is doing.
So a developer sends a second request to find out: "Is the address valid?" The model answers politely that it has not been given an address. It has no trace of the email it read ten seconds ago.
Both surprises have the same cause, and every part of an agent system exists because of it. This chapter looks at one call, slowly: what goes in, what comes back, and what is gone the moment it ends.
A hosted model is a function over HTTP. It reads one request, writes one response, and keeps nothing. Whatever must last longer than one call has to live in software around the call.
1. What the team is talking to#
The use case is the one from the start page: a broker asks an insurer for a homeowners quote by email, and the system must answer with a priced quote, a question, or a referral to a person. The email is one line: "please quote homeowners for 18 Gulf Shore Blvd, Naples FL, built 1998, 2,400 sq ft".
The team wants a language model to do the first pass. On this site, model means the language model alone: a program that reads text and writes text. The team does not run it. A vendor runs it on its own servers and offers it as an HTTP endpoint (a URL that accepts requests). The program posts JSON to that URL and receives JSON back.
That is all the model is from the outside. There is no session to open, no connection to keep, no object that represents "the conversation". For Claude models the endpoint is called the Messages API, and its reference describes it as usable for "single queries or stateless multi-turn conversations". The word that matters is stateless. Section 4 makes it concrete. First, the request.
2. The request: everything the model will know#
Whatever the model should take into account must be inside the request body. That is the rule that shapes the request. The model cannot look anything up, and it has no earlier call to draw on. If a fact is not in the JSON, it does not exist for this call.
The script below is examples/quote-email/curl_example.sh, copied whole. It shows the smallest useful request for the quote example: instructions, one tool the model may ask for, and the broker's email. It was not run while this site was built. It follows the request shape in the API reference, which was read on 2026-11-16. A successful run would prove that the key, the headers and the body are accepted. It would not prove the answer is right.
#!/bin/sh
# One call to the Messages API. Content in; content and control signals out.
# The key comes from the environment. Never paste a real key into a file.
# The documented shape of the reply is in sample_response.json (illustrative).
curl https://api.anthropic.com/v1/messages \
-H "content-type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2026-10-21" \
-d '{
"model": "claude-opus-5-5",
"max_tokens": 16000,
"system": "You triage inbound homeowners quote emails for an insurance carrier.",
"tools": [{
"name": "verify_address",
"description": "Check that a street address exists.",
"input_schema": {
"type": "object",
"properties": {"address": {"type": "string", "description": "Street, city and state"}},
"required": ["address"]
}
}],
"messages": [
{"role": "user", "content": "Please quote homeowners for 18 Gulf Shore Blvd, Naples FL, built 1998, 2,400 sq ft."}
]
}'
Read it from the top. It is a POST to /v1/messages with three headers. x-api-key says who is paying. anthropic-version pins the API version, so that a later change to the API does not change what this program receives. content-type says the body is JSON. The key is read from an environment variable and never written into the file.
The body has five fields, and each one answers a question the model cannot answer for itself.
Which model, and how long may it write? model names the model. max_tokens is a ceiling on the length of the reply. Length is counted in tokens (pieces of words; section 5 explains them). The ceiling is a safety limit, not a target: the reference notes that a model may stop before reaching it.
What is its job? system carries the system prompt: standing instructions that apply to the whole call. Here it is one sentence that says the model triages quote emails for an insurer. It sits outside the conversation because it is not something the broker said. It is what the insurer's program says about how to treat everything else.
What may it ask for? tools is a list of tool definitions. Each has a name, a description and an input_schema: a JSON Schema that says which arguments the tool takes. This definition does not give the model the ability to verify an address. It only tells the model that such a function exists on the caller's side, and what a well-formed request for it looks like. Tools is the home of schemas and of where tools run.
What was said so far? messages is the conversation as a list. Each entry has a role, user or assistant, and a content. This first request has one entry: the broker's email, in the user role. The role name is historical. It means "the caller's side", not "a person": the insurer's program wrote this entry.
The table collects the five fields. The descriptions in the third column follow the API reference.
| Field | In this request | What it is for |
|---|---|---|
model | claude-opus-5-5 | Which model answers |
max_tokens | 16000 | The most tokens the model may generate before it is stopped |
system | One sentence about triage | The system prompt: context and instructions for the whole call |
tools | One definition, verify_address | Tools the model may ask the caller to run |
messages | One user entry, the email | The prior turns of the conversation, in order |
One thing is missing from the table, and its absence is the point of this chapter. There is no conversation id, no session token and no "continue from last time" field in this request. Nothing in it refers to any earlier call.
The three headers, and what the documentation says about eachRead on 2026-11-16; sources in the fact log
| Header | In the example | What the documentation says |
|---|---|---|
x-api-key | $ANTHROPIC_API_KEY | The API key. The API overview also lists Authorization: Bearer <token> and calls x-api-key a legacy fallback that is still supported. The curl examples in the documentation use x-api-key. |
anthropic-version | 2026-10-21 | The API version. The versioning page lists 2026-10-21, 2023-06-01 and 2023-01-01, and recommends the latest. |
content-type | application/json | Required. The body is JSON. |
A version is a contract about the request and response shape. The versioning page says that within a version, existing input and output parameters are preserved. A new version may remove something. Version 2026-10-21 removes the request parameters temperature, top_p and top_k: a request that sends one of them returns a 400 error.
Every response also carries a request-id header. It is the value to log, because it is what the vendor's support asks for.
3. The response: content, and signals for the caller#
The reply is one JSON object. It holds two different kinds of thing, and telling them apart is most of the skill of reading it. Some of it is content: what the model wrote. The rest is control signals: fields that tell the calling program what state the reply is in and what to do next.
The file below is examples/quote-email/sample_response.json. It is illustrative. It was written by hand from the API documentation to show the documented shape of a reply to the request above. It was not captured from a real call: the ids, the text and the token counts are invented. A real reply can carry more blocks and more fields, for example a stop_details field next to stop_reason.
{
"_note": "ILLUSTRATIVE. Hand-written from the Messages API documentation to show the response shape for curl_example.sh. Not captured from a real call: ids, text and token counts are invented, and a real reply can carry extra blocks (for example a thinking block) and extra usage fields.",
"id": "msg_01EXAMPLEEXAMPLEEXAMPLE",
"type": "message",
"role": "assistant",
"model": "claude-opus-5-5",
"content": [
{
"type": "text",
"text": "I'll verify the address first."
},
{
"type": "tool_use",
"id": "toolu_01EXAMPLEEXAMPLEEXAMPLE",
"name": "verify_address",
"input": {
"address": "18 Gulf Shore Blvd, Naples FL"
}
}
],
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 412,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 58
}
}
The content is a list of blocks, not a string#
A first-time reader expects content to be text. It is a list, because a model can return more than one kind of thing in one reply. Each item is a content block with a type.
This reply has two blocks. The first has type text: the sentence from the opening of this chapter. The second has type tool_use, and it is not prose. It holds the name of a tool from the request, an input object that follows that tool's schema, and an id that identifies this one request for a tool.
A program that prints only the text blocks sees "I'll verify the address first." and nothing else. That is the first surprise from the opening. The model did say what it wanted. It said it in a block the program was not reading.
The stop reason says why the model stopped writing#
Every reply ends for a reason, and stop_reason names it. Here the value is tool_use: the model stopped because it is waiting for the result of a tool. The reply is not an answer to the broker. It is a request to the caller.
Had the model simply finished, the value would be end_turn. Had it run into the ceiling from the request, the value would be max_tokens, and the text would be cut off mid-thought. These three cases need three different reactions from the program, and the text alone does not say which case it is. A reply that ends in a full sentence can still be a truncated one.
That is why this site calls stop_reason a control signal. It is written for code, not for a reader. The first line of any program that handles a reply should branch on it.
stop_reason | What happened | What the caller does |
|---|---|---|
end_turn | The model completed its response | Use the content |
tool_use | The model asks for a tool | Run the tool and send the result in a new call. The loop is about exactly this. |
max_tokens | The reply reached the max_tokens limit | Treat the content as cut off |
All seven documented stop reasonsFrom the stop-reasons page of the documentation, read on 2026-11-16
stop_reason | The documentation's description | What it means for the caller |
|---|---|---|
end_turn | "Claude completed its response naturally." | The normal end |
max_tokens | "The response reached your max_tokens limit." | The content is truncated |
stop_sequence | "Claude emitted one of your stop_sequences." | The caller asked for this stop; stop_sequence holds the text that matched |
tool_use | "Claude is calling a tool." | Run the tool and return the result |
pause_turn | "A server-tool loop reached its iteration limit." | Send the assistant content back to continue. This applies to tools the vendor runs. |
refusal | "Claude declined to respond." | Do not treat the content as an answer |
model_context_window_exceeded | "The response filled the model's context window." | Treat the response as truncated |
The documentation's phrase is that the model "is calling a tool". This site words it differently on purpose: the model asks, the harness decides and calls, the tool does. For a tool defined in the request, as verify_address is, nothing runs until the caller runs it.
The usage block is the meter#
usage counts what this call consumed, in tokens. input_tokens is what the model read. output_tokens is what it wrote. The two fields with cache in their names are also input: the documentation says the input count is split across the three fields. They are zero here, and Context explains when they are not.
The numbers in the sample are invented. What matters is the shape of the bill. The caller pays for input on every call, and input is everything in the request: the documentation counts the system prompt, every message and the tool definitions. So the one-line email is not a one-line input.
The diagram puts the two halves side by side.
Diagram: One call · what goes in, what comes back
Read left to right. The left column is the request the caller builds. The middle is one run of the model. The right column is the reply. Two of the three boxes on the right are signals for a program, not text for a person. Click any box.
- Request · what the caller sends
- system: The system prompt: standing instructions for the whole call.
- tools: Definitions of tools the model may ask the caller to run: name, description, input schema.
- messages: The whole conversation so far, as a list of user and assistant entries.
- model + max_tokens: Which model answers, and the most tokens it may generate.
- The model · one inference step
- HTTPS POST /v1/messages: One request to the Messages API. No session is opened and none is kept.
- Reads the request, writes a reply: The model reads the context in this request and generates a reply. It can read nothing else.
- Keeps nothing: When the reply is sent, nothing about this call remains on the model's side.
- Response · what comes back
- content: A list of blocks: text the model wrote, and tool_use blocks that ask for a tool.
- stop_reason: Why the model stopped writing. The caller branches on this before it reads the text.
- usage: Tokens read and tokens written in this call. The meter for cost.
Read the right-hand column again. Two of its three boxes are not for a person to read. A model reply is a message to a program, and the program has to act on it.
4. Send the same request twice#
Now the second surprise. Suppose the program sends the request from section 2 twice, one second apart.
The endpoint treats them as two unrelated requests. Each one is read from the first byte, answered and forgotten. The second reply does not say "as I mentioned". It asks for verify_address again, as if for the first time, because for the model it is the first time.
The two replies may also differ in wording. The vendor's glossary says that identical inputs might produce different outputs across API calls. So the caller cannot treat a repeated request as a way to get the same reply back.
This is what stateless means. Nothing about a call survives on the model's side: not the email, not the reply, not the fact that a call happened. What a vendor may log or store for billing and safety is a separate matter, and the model does not read from it.
Now the developer's follow-up from the opening makes sense. It is shown here as a mistake. This body is illustrative and was not sent:
{
"model": "claude-opus-5-5",
"max_tokens": 16000,
"messages": [
{"role": "user", "content": "Is the address valid?"}
]
}
It is a valid request, and that is the trap. The API accepts it. But the model receives nine words and nothing else. No email, no address, no system prompt, no tool. The mistake is the assumption that the endpoint is holding the conversation.
The rule that fixes it: the caller keeps the conversation and sends all of it, every time. A correct second request repeats the system prompt and the tools, repeats the email as the first message, adds the model's own reply as an assistant message, and only then adds what is new. The model reads its own earlier words as input. That is how it "remembers" them.
Diagram: Two calls · the caller carries the conversation
Time runs down the page. The endpoint answers each request from what that request contains. Between two calls, only the caller holds anything. The second request is larger because it contains the first.
- The insurer's program to Model endpoint: Call 1: system, tools, the email. Everything the model will know is in this request.
- Model endpoint to The insurer's program: Reply 1: a tool request. The stop reason is tool_use. The call is over.
- Model endpoint: Nothing is kept. The endpoint holds no conversation. The email and the reply are gone on its side.
- The insurer's program: Appends reply 1 to its list. The only copy of the conversation is in the caller's memory.
- The insurer's program to Model endpoint: Call 2: system, tools, the email, reply 1, what is new. The caller sends the whole history again. The model reads its own earlier reply as input.
- Model endpoint to The insurer's program: Reply 2. Answered from the contents of call 2 alone. Input tokens are higher than in call 1.
The miniature shows this with code that runs. The function below is call_model from examples/quote-email/agent_minimal.py. It is the same request as the curl script, written with the Python standard library. Notice that it takes the whole messages list as its argument. It has nothing else to send.
def call_model(messages: list[dict]) -> dict:
"""One HTTPS POST. The model keeps nothing, so every call carries the full history."""
body = {"model": "claude-opus-5-5", "max_tokens": 16000, "tools": TOOLS, "messages": messages}
request = urllib.request.Request("https://api.anthropic.com/v1/messages", json.dumps(body).encode(), {
"x-api-key": os.environ["ANTHROPIC_API_KEY"],
"anthropic-version": "2026-10-21",
"content-type": "application/json"})
with urllib.request.urlopen(request, timeout=120) as reply:
return json.load(reply)
The run below uses the scripted fake model, so it needs no key and no network. --verbose prints each model response. The script makes two calls, because it contains the loop that the next chapter reads. For this chapter, look at one number only: input_tokens in each response.
cd examples/quote-email
/opt/homebrew/bin/python3 agent_minimal.py --fake --verbose
{"id": "msg_fake_01", "type": "message", "role": "assistant", "model": "claude-opus-5-5", "content": [{"type": "tool_use", "id": "toolu_fake_01", "name": "check_underwriting_rules", "input": {"address": "18 Gulf Shore Blvd, Naples FL", "year_built": 1998, "sqft": 2400}}], "stop_reason": "tool_use", "stop_sequence": null, "usage": {"input_tokens": 109, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "output_tokens": 42}}
{"id": "msg_fake_02", "type": "message", "role": "assistant", "model": "claude-opus-5-5", "content": [{"type": "text", "text": "{\"decision\": \"quote\", \"missing_fields\": [], \"reasons\": [\"eligible; flood zone AE\"], \"premium_band\": \"high\"}"}], "stop_reason": "end_turn", "stop_sequence": null, "usage": {"input_tokens": 210, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "output_tokens": 37}}
{"decision": "quote", "missing_fields": [], "reasons": ["eligible; flood zone AE"], "premium_band": "high"}
The first call reports 109 input tokens. The second reports 210. Nothing new arrived from the broker between the two. The second request is larger because it contains the first request again, plus everything that happened since.
Be clear about what this run proves. It proves that the caller in this file resends the history: the request body grew. It does not measure real token counts. The fake model has no tokenizer and estimates a token as four characters of JSON. This miniature also has a different single tool, check_underwriting_rules, from the curl script.
5. Resending has a size, and the size has a limit#
If every call carries the whole conversation, two questions follow. How big can a request get? And what does the size cost?
Both are measured in tokens. A model does not read characters or words. Its input is first cut into small units, and the vendor's glossary says a token can correspond to a word, a part of a word, a character or even a byte. A short common word is often one token. A street address or a policy number breaks into several. Everything about a call is counted in this unit: the ceiling in max_tokens, the numbers in usage, the bill, and the limit below.
The limit is the context window. The context is everything sent to the model in one call. The context window is the most the model can hold at once: the documentation defines it as all the text the model can reference when generating a response, including the response itself. So input and output share one budget.
For the model in the example, claude-opus-5-5, the models overview lists a context window of 1M tokens and a maximum output of 128K tokens, as read on 2026-11-16. One email is nowhere near that. A conversation that grows with every call gets there in the end, and two things happen at the edge:
- If the input alone is too long, the API rejects the request with a 400 error, "prompt is too long". Nothing is generated.
- If the input fits but the reply runs into the limit while it is being written, generation stops with
stop_reasonset tomodel_context_window_exceeded. On Claude 4.5 models and newer, the documentation says the API accepts such a request and stops this way.
A caller does not have to guess the size. A second endpoint, POST /v1/messages/count_tokens, takes the same inputs as a message request and returns input_tokens without running the model. The documentation says it is free to use, subject to rate limits, and that the count is an estimate.
The limit is rarely the first problem. The cost is. A conversation of ten calls does not cost ten times the first call. Each call resends everything before it, so the tenth call pays again for the nine before. This is the reason a later chapter treats what enters the context as the main design decision. The remedies have names: prompt caching, which lowers the price of a repeated prefix, and compaction, which replaces old turns with a summary. Their home is Context. Here it is enough to know why they exist: the model keeps nothing, so the caller sends everything, so everything is paid for again.
6. What one call cannot do#
It is worth collecting what the team has learned, because each item is a gap that some later piece of software has to fill. None of them is a defect of the model. They follow from the shape of the call.
It cannot remember. Section 4 showed it. Something must keep the conversation between calls, and keep it through a crash if the work matters.
It cannot act. The tool_use block is a request. The address is not verified until some code on the caller's side runs verify_address. The same holds for every tool the caller defines: the model asks, the harness decides and calls, the tool does.
It cannot continue. After a tool_use reply the model is not waiting. The call is over. If the result of the tool should reach the model, somebody has to make a new call that contains it.
It cannot be trusted to be complete or well-formed. A reply can be cut off at max_tokens. It can be a refusal. Its text can claim to be a decision without being valid JSON. The caller has to check.
It can fail outright. The endpoint is a network service. The errors page of the documentation lists, among others, status 429 for a rate limit and 529 when the API is temporarily overloaded.
| What one call lacks | Why | Where this site deals with it |
|---|---|---|
| Memory between calls | The endpoint is stateless | Context, and Control flow and state |
| The ability to act | A tool request is only a message | The loop, then Tools |
| A next step | A call ends when the reply is sent | The loop |
| A checked output | The reply is whatever the model wrote | Guardrails |
| A guarantee of an answer | Rate limits, overload, truncation | Reliability and cost |
Every row of this table is work for the software around the call. That software is the harness, and the rest of the site is about it.
7. Who keeps the conversation?#
The model does not. So something else does, and there are only a few places it can be. This is the first time the site's standing question appears: who runs this job, you, a library, or a vendor?
Your own code can keep it. That is what agent_minimal.py does: a Python list named messages, appended to after every call and sent whole with the next one. It is the plainest arrangement and nothing is hidden. It also means that if the process dies, the list dies with it. The larger miniature, harness.py, writes a checkpoint to SQLite after every step for that reason. Control flow and state covers that.
A library in your process can keep it. A provider SDK wraps the HTTP call and leaves the list to you. An agent SDK or a framework holds the list for you and resends it. The request on the wire is the same one as in section 2. The ladder sorts these layers.
A vendor can keep it. Some APIs store the conversation on the server, and the next request refers to it by an id. The OpenAI Responses API offers this as an option, with previous_response_id and durable conversation objects. That does not make the model remember. A different program holds the state, and each inference step (one run of the model) still receives the whole context.
| Where the conversation lives | Example | Who runs the job |
|---|---|---|
| In the caller's code | Messages API with a messages list, as in agent_minimal.py | YOU BUILD |
| In a library inside the caller's process | An agent SDK or framework over the same API | BUILT IN to the library; the process and its storage are still yours |
| On the vendor's servers | OpenAI Responses API with previous_response_id; the baseline fact check records that responses are stored for 30 days by default | MANAGED, as an option |
The table is not a ranking. It says where to look when a conversation is lost, grows too large or must be deleted: at whoever holds the list.
One more note on the word HTTP. Hosted models are commonly reached this way, and this site uses that case throughout. A model that runs on your own machine is a function call inside your process. The shape does not change: content in, content and signals out, nothing kept.
What is, and is not, run on this pageOne fake-model run; no live API call
| Shown on this page | Status |
|---|---|
curl_example.sh | Follows the documented request shape. Not run against the real endpoint. Its syntax is checked with bash -n. |
sample_response.json | Illustrative. Hand-written from the documentation; ids, text and token counts are invented. |
| The "Is the address valid?" request | Illustrative. A labelled mistake; not sent. |
agent_minimal.py --fake --verbose | Run on 2026-11-16. The output is pasted unchanged. The model is the scripted fake; its token counts are estimates. |
| Field names, headers, the endpoint, the model id, stop reasons, limits | Read in the vendor's documentation on 2026-11-16 and listed in the fact log of this site. |
The behavior described in section 4, two identical requests answered independently, follows from the documented design of the API. It was not observed in a live call for this page.
8. The reply asked for something, and nobody is there#
Go back to the reply in section 3 with what you now know. The team can read it completely: a text block, a tool_use block that names verify_address and gives the address, a stop reason that says the model is waiting, and a meter.
And still nothing in one call can answer it. The model has stopped. The endpoint has forgotten. The address is unverified, and the only thing in the world that knows a tool was asked for is the program holding this JSON. If that program runs the tool, puts the result into the list and calls again, the work continues. If it does not, the sentence "I'll verify the address first" stays a promise.
A single call is a pure function: the same kind of thing every time, with no past and no hands. Everything that looks like an agent comes from what the caller does between calls. The smallest version of that is a loop, and the next chapter builds it in about thirty lines.