The team's harness is finished. harness.py is about 450 lines of Python with no dependencies. It reads the broker's email, runs three tools, stops at the approval gate, continues after a crash and never sends the reply twice. Then a new engineer joins, reads the file in an afternoon, and asks the question every such file gets: why did you not use a framework?
Nobody has a good answer ready. "We wanted to understand it" is true, and it does not answer the question. The question has two halves. Which of these 450 lines would a framework have written for us? And which would we have had to write anyway?
The ladder placed frameworks on rung 4 and graph runtimes on rung 5, and showed one fragment of each. This chapter opens those two rungs: ten products, read with the eight jobs as the checklist.
A framework is the half of a harness that is the same in every company, written by someone else. The half that names your business was never in it. Reading a framework means finding the line between the two halves.
1. Sort the 450 lines by one question#
Take each function in harness.py and ask: is this code about insurance, or about agents? Code about agents would look the same in a harness for travel bookings or support tickets. Code about insurance would not survive the move.
| Job | The same in any agent | Only true for this insurer |
|---|---|---|
| 1 Context assembly | The order of the request body in assemble_context | SYSTEM_PROMPT, RULE_DOCS, which rules retrieve_rules may fetch for which role |
| 2 Tools | _schema, TOOL_SCHEMAS, the guard in run_tool_guarded | What check_underwriting_rules decides |
| 3 Control flow | run, model_step, tool_step | Nothing. The loop has no insurance in it. |
| 4 State | checkpoint, load_checkpoint | Which database, and how long a quote is kept |
| 5 Identity, approval | The pause when authorize returns "wait" | ROLE_TOOLS, authorize, enforce_policy |
| 6 Output contract | Parse, check, ask again: validate_decision, repair_prompt | The fields of the decision: DECISIONS, BANDS |
| 7 Reliability, cost | call_with_retry, pick_model, over_budget | The numbers: MAX_TURNS = 8, the token budget. The key in send_once. |
| 8 Observability | trace | golden.jsonl: what a correct quote is |
The middle column is what a framework sells. The right column is what no framework can contain, because its authors have never seen an underwriting rule. Job 3 is the clearest row. The loop is pure agent code, which is why every product in this chapter ships it.
Diagram: The 450 lines, sorted · what a framework writes, what a graph runtime adds, what stays with you
Three groups. The first is code that looks the same in every agent, so an agent framework ships it. The second is what a graph runtime adds on top: a run that is saved and can continue in another process. The third is the part that names the insurer. No product on this page contains it. Click any box.
- Rung 4 · a framework writes
- The loop: One constructor replaces the three functions that decide the next step, call the model and run the tools.
- Tool wiring: The framework reads a function's name, type hints and docstring and builds the tool schema the model sees.
- Retries and fallback: Code that wraps the model call: try again after a rate limit or a server error, or switch to a second model.
- Output parsing: You pass a typed class. The framework asks for that shape, validates the answer and can ask again when it fails.
- Tracing hooks: The loop emits an event for each model call and each tool call, so a tracing product can store them.
- Rung 5 · a graph runtime adds
- Saved state: The runtime writes the state of the run to a store as the run proceeds. You choose the store and operate it.
- Pause for a person: A step can stop the run and hand a question to a person. Nothing has to stay in memory while the answer is pending.
- Resume: A different process loads the saved state and continues. Some code runs again, so side effects must be safe to repeat.
- Stays with you on both rungs
- Rules and prompts: The system prompt, the underwriting rules and the body of every tool. A framework has never seen them.
- Who may do what: Which caller may use which tool, and which action needs a person. Frameworks ship the pause, and leave the policy and the caller's identity to you.
- The numbers: How many turns and how many tokens a run may use. A framework has a default or none. The right number depends on the task.
- Safe side effects: Making a repeated step harmless. Every runtime that resumes work says this is your job.
- What correct means: The cases that define a correct answer, and the score that says whether a change helped.
The diagram has three groups, and the next two sections take the first two in turn. Keep the third group in view while reading any product page: it is the list of things you will still be writing after the install.
2. The repeated half: five things every team writes again#
Five pieces of the middle column show up in every agent project. Each has a function in the miniature, and each has a packaged form on rung 4.
The loop. run decides the next step, model_step makes one model call, tool_step runs the tools the model asked for. A framework gives you one constructor that returns an object with a run method. The model still only asks. The framework's loop decides and calls. Your function does the work.
Tool wiring. The miniature writes each tool twice: once as a Python function, once as a JSON Schema in TOOL_SCHEMAS, and a dictionary connects the two. A framework reads the function's signature and docstring and builds the schema itself. Section 5 shows this with one function used in nine fragments.
Retries. call_with_retry is a dozen lines: retry status 429 and 5xx, wait longer each time, honour a retry-after header. Frameworks package this, though not all of them turn it on, as section 6 shows.
Output parsing. validate_decision parses the final text and repair_prompt asks once more when it fails. A framework takes a typed class and does both.
Tracing hooks. trace writes one JSON line per model response and per tool call. A framework emits such events from inside its loop, usually in the OpenTelemetry format, so a tracing product can store them.
Look at where the miniature puts the retry. This line is quoted from model_step in harness.py:
response = call_with_retry(model, lambda name: assemble_context(caller, state, name))
The model call is not made directly. It is handed to another function, which runs code before it, may run it several times, and handles its failure. tool_step does the same for tools: authorize runs first and can replace the call with a denial. A harness is full of such wrapping points, and once a framework owns the loop, you can no longer reach them by editing the loop. So frameworks expose them as a list you fill in.
That list is called middleware: a function the framework's loop calls around each model call or each tool call, which can inspect the request, change it, retry it, or stop it. LangChain's documentation says middleware lets you "more tightly control what happens inside the agent". Microsoft Agent Framework says it lets you "intercept, modify, and enhance agent interactions at various stages of execution". The retry, the fallback model, the call limit and the approval pause on rung 4 are all delivered this way. Middleware is how a hidden loop stays configurable.
3. What a graph runtime adds: a run that outlives its process#
A rung 4 loop lives inside one process. The quote example has a step that no single process can finish: a person must approve the reply, and may answer in two days. So the run must be saved, the process must be free to exit, and a different process must continue later from what was saved. Control and state explains checkpoints and resuming in full. Here is the short form, in the miniature, because it is the work a graph runtime takes over.
The two commands are from the miniature's README. They show a run that stops at the gate and a second process that continues it. A successful pair proves that everything the run needs is in runs.sqlite, not in memory. It was run against the scripted model, so the token counts are not measurements.
/opt/homebrew/bin/python3 harness.py --fake
/opt/homebrew/bin/python3 harness.py --fake --resume run-690310f5 --approve
send_reply is waiting for a human. Continue with: --resume run-690310f5 --approve
{"run_id": "run-690310f5", "status": "waiting_approval", "decision": null, "reply": "waiting_approval", "turn": 4, "tokens": 3020}
{"run_id": "run-690310f5", "status": "done", "decision": {"decision": "quote", "missing_fields": [], "reasons": ["eligible; flood zone AE"], "premium_band": "high"}, "reply": "sent", "turn": 5, "tokens": 4013}
(The real output is indented JSON. It is shown here on one line per run.)
In the miniature this costs three functions (checkpoint, load_checkpoint, send_once) and two tables. A graph runtime packages them. You describe the work as nodes and edges, and the runtime saves the state as the run proceeds, so a run can pause and resume. The cause it removes is the process boundary: a crash, a deploy or a two-day wait no longer ends the run.
It also hands you a rule, and every product that resumes work states it in its own words. When a run continues, some code runs again.
| Product | What its documentation says about resuming |
|---|---|
| LangGraph | "the runtime restarts the entire node from the beginning". A retry "does not make external side effects happen exactly once." |
| LlamaIndex Workflows | "Resume is at-least-once, and step side effects need to be safe to repeat." |
| Google ADK | Tools "may run more than once when resuming a workflow." |
| OpenAI Agents SDK | "Approval permits a tool call to execute; it does not guarantee exactly-once execution." |
| Temporal | "A Workflow Definition must be deterministic to support replay." Model calls and other outside work go into Activities. |
Five vendors, one message: the runtime saves progress, and making a repeated step harmless is your job. The miniature does that with send_once and an outbox table. On a graph runtime you still write that function. Reliability and cost is its home.
The same pause on a graph runtime, run here without a modelThree nodes, one interrupt, an in-memory saver
To see the pause without any model, a three-node graph was run for this page with langgraph 1.2.22: a rules node that calls check_underwriting_rules, an approval node that calls interrupt, and a send node. The first invoke returned at the gate. The second, graph.invoke(Command(resume="approve"), config) with the same thread_id, finished the run.
paused at: ('approval_gate',)
waiting for: {'draft': {'outcome': 'quote', 'premium_band': 'high', 'reasons': ['eligible; flood zone AE']}}
after resume: sent | checkpoints saved: 5
This used InMemorySaver, which keeps checkpoints in RAM, so it proves the mechanism and nothing about durability. The documentation says to "use a durable checkpointer in production" and names PostgresSaver and MongoDBSaver as "ideal for using in production". Both calls ran in one process here. The fragment for this pattern is on The ladder.
4. Ten products as eight jobs each#
The table translates each product into the eight jobs. The words are the ones this site uses everywhere. BUILT IN: the product ships it. INTEGRATES: a named partner does it. YOU BUILD: the documentation leaves it to your code. NOT ITS JOB: this site's reading of the product's scope. "The pages read do not say" means exactly that. It never means the feature is absent.
Every durability cell was read on the vendor's page on 2026-11-16. The full matrix, with SDKs and managed platforms, is on Build or adopt.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
LangChain create_agent · agent framework | BUILT IN: system_prompt, middleware. Summarization runs only if you set a trigger. | BUILT IN: tools from functions. The shell tool runs on the host unless you pick DockerExecutionPolicy. | BUILT IN: the loop, on LangGraph | BUILT IN once you pass a checkpointer | BUILT IN: HumanInTheLoopMiddleware. Caller identity: the pages read do not say. | BUILT IN: response_format | BUILT IN: retry, fallback and call-limit middleware | INTEGRATES: LangSmith |
| LangGraph · graph runtime | BUILT IN: state, store. Prompts are yours: it "does not prescribe prompts". | BUILT IN: a prebuilt ToolNode class. The quickstart writes the tool node by hand. Sandbox: the pages read do not say. | BUILT IN: the graph | BUILT IN: checkpointers for memory, SQLite, PostgreSQL, MongoDB, Cosmos DB | BUILT IN: interrupt. Identity: the pages read do not say. | The pages read do not say | BUILT IN: RetryPolicy, node timeouts. Token budget: the pages read do not say. | INTEGRATES: LangSmith |
| LlamaIndex · agent framework, data-centric | BUILT IN: retrieval over your data, memory classes | BUILT IN: tools from functions. Sandbox: not re-read today. | BUILT IN: event-driven Workflows. FunctionAgent is a pre-built one. | YOU BUILD the saving: ctx.to_dict() and your own store. INTEGRATES: DBOS, to survive a crash. | BUILT IN: InputRequiredEvent, HumanResponseEvent. Identity: the pages read do not say. | BUILT IN: output_cls | BUILT IN: retry_policy on steps. Agent turn limit: the pages read do not say. | BUILT IN: instrumentation, evaluation modules. INTEGRATES: Arize Phoenix, Langfuse, Opik. |
| CrewAI · agent framework | BUILT IN: role, goal and backstory templates, memory. Summarizes at the context limit by default. | BUILT IN: tools, MCP. Code execution is deprecated. | BUILT IN: Crews and Flows | BUILT IN: checkpoints to JSON files or SQLite; @persist on Flows. Resuming is a call you make. | BUILT IN: human_input, @human_feedback. Identity: YOU BUILD. | BUILT IN: output_pydantic, output_json | BUILT IN: max_iter, max_retry_limit, max_rpm. Fallback: the pages read do not say. | MANAGED: CrewAI AMP tracing. INTEGRATES: Langfuse, Arize Phoenix and others. |
| Pydantic AI · agent framework, typed | BUILT IN: instructions, typed dependencies | BUILT IN: tools, MCP. Sandbox: not confirmed today. | BUILT IN: the loop, pydantic-graph | YOU BUILD: you store the message history. INTEGRATES: nine durable execution engines. | BUILT IN: requires_approval. Identity: YOU BUILD. | BUILT IN: output_type, with a validation retry | BUILT IN: UsageLimits, ModelFallback, retries | BUILT IN: OpenTelemetry, Pydantic Evals. INTEGRATES: Logfire. |
| Microsoft Agent Framework · agent framework | BUILT IN: context providers. Compaction is experimental in Python. | BUILT IN: tools, MCP. Shell tools: a beta package. | BUILT IN: agents and graph workflows | BUILT IN: AgentSession, which you serialize; workflow checkpoints to memory, file or Cosmos DB | BUILT IN: approval_mode. Identity: YOU BUILD. | BUILT IN: response_format | YOU BUILD: retry and fallback as middleware. Tool-call limits default to unlimited. | BUILT IN: OpenTelemetry. Evals: the pages read do not say. |
| OpenAI Agents SDK · agent SDK | BUILT IN: instructions, sessions, an input filter | BUILT IN: function tools, hosted tools, MCP, sandbox agents | BUILT IN: the loop, handoffs | BUILT IN: sessions (SQLite, Redis, SQLAlchemy, MongoDB and others), RunState. INTEGRATES: Dapr, Temporal, Restate, DBOS. | BUILT IN: guardrails, needs_approval. Authorization: YOU BUILD, inside the tool. | BUILT IN: output_type | BUILT IN: max_turns, usage tracking, opt-in retries. Fallback: the pages read do not say. | BUILT IN: tracing, on by default. Evals: not in the SDK pages read. |
| Google ADK · agent framework | BUILT IN: instruction templates, compaction. Context caching is experimental. | BUILT IN: tools, MCP, OpenAPI | BUILT IN: graph workflows | BUILT IN: session services for memory, a database or Agent Platform; resumable workflows | BUILT IN: require_confirmation, tool authentication | BUILT IN: output_schema | BUILT IN: max_llm_calls, retry on graph nodes. Python fallback: the pages read do not say. | BUILT IN: logging, OpenTelemetry traces, adk eval |
| Strands Agents · agent SDK | BUILT IN: conversation managers | BUILT IN: tools, MCP, Docker and SSH sandboxes | BUILT IN: the loop, optional limits | BUILT IN: session managers for files and S3. Durable execution: the pages read do not say. | BUILT IN: HumanInTheLoop, CedarAuthorization | BUILT IN: structured_output_model | BUILT IN: retries on throttling, limits you pass | BUILT IN: OpenTelemetry traces |
| Temporal · durable execution platform | NOT ITS JOB | NOT ITS JOB: a tool call is one of your Activities | BUILT IN: Workflows as code | BUILT IN: Event History and replay | NOT ITS JOB. A Signal lets a Workflow wait for a person. | NOT ITS JOB | BUILT IN: Activity retries, unlimited attempts by default | BUILT IN: metrics, tracing, logging |
Three things are worth taking from the table.
Column 3 says BUILT IN in every row. That is what makes a product belong in this chapter. The role labels are looser than they look: the OpenAI Agents SDK and Strands Agents are SDKs by name, and both ship sessions, approval and tracing, which is rung 4 work. Read the cells, not the label.
Column 5 splits down the middle of its own name. The pause for a person is built in almost everywhere. Establishing who the caller is stays outside: no page read shows a framework authenticating the end user, and four vendors hand that work back in so many words. CrewAI: "applications must still authorize access to persisted sessions." Microsoft: "verify the authenticated user or tenant before resuming a conversation." Pydantic AI calls its approval step "not an authorization boundary against an untrusted client". OpenAI's context page tells you to "enforce those decisions inside the tool implementation". That is ROLE_TOOLS and authorize from section 1, still in the right column. Guardrails is the home of that work.
Column 4 hides two different promises. One is "the next message remembers the last one", which a session store gives you. The other is "the run survives a crash in the middle", which is durable execution. Products answer the second in three ways: their own saving mechanism (LangGraph, Microsoft Agent Framework workflows, CrewAI, Google ADK), a partner engine (Pydantic AI, the OpenAI Agents SDK, LlamaIndex with DBOS), or nothing on the pages read (Strands Agents). LlamaIndex is plain about it: "There is no built-in checkpointer to enable."
5. One tool function, nine fragments#
Each tab below builds the same thing: an agent with one tool. The tool is this function, written once.
def check_underwriting_rules(address: str, year_built: int, sqft: int) -> dict:
"""Apply the underwriting rules to a property. Returns the outcome and the reasons."""
return apply_rules(address, year_built, sqft) # the insurer's own code
INSTRUCTIONS = "You triage homeowners quote emails. Check the underwriting rules before you answer."
Every fragment follows the vendor's documentation as read on 2026-11-16. Each agent was also constructed for this page in a scratch environment, with a placeholder in place of an API key, to confirm that the imports and parameters exist in the version installed. No fragment called a model. A constructed agent proves the names are right. It proves nothing about how the agent answers. The lines that would call a model are marked.
LangChain
What it is. "LangChain provides create_agent: a minimal, highly configurable agent harness." Its agents run on LangGraph. Deep Agents, from the same vendor, "adds planning, a virtual filesystem for context, subagent spawning, and long-term memory on top of LangChain and LangGraph".
Jobs it runs for you. The loop, tool schemas from plain functions, structured output through response_format, and a catalogue of middleware: retry, fallback, call limits, summarization, human approval.
What stays with you. Each of those is a line you add, and the limits inside ModelCallLimitMiddleware are documented as "Defaults to no limit." History is kept only if you pass a checkpointer and run its database. Tracing needs a LangSmith key.
# Adapted from the official documentation. Constructed here (langchain 1.5.0); the last line was not run.
from langchain.agents import create_agent
agent = create_agent(
model="anthropic:claude-opus-5-5",
tools=[check_underwriting_rules],
system_prompt=INSTRUCTIONS,
)
result = agent.invoke({"messages": [{"role": "user", "content": email_text}]}) # calls the modelLangGraph
What it is. "LangGraph is a low-level orchestration framework and runtime for long-running, stateful agents. It does not prescribe prompts or agent architecture."
Jobs it runs for you. Running the graph, saving it ("LangGraph creates a checkpoint at each super-step boundary"), pausing with interrupt, and retrying a node.
What stays with you. The loop itself. You draw it: a model node, a tool node, and an edge back. The tool node can be the prebuilt ToolNode class from langgraph.prebuilt; the quickstart writes it by hand, and so does the fragment below. You also choose the checkpointer, and you keep side effects safe to repeat.
# Adapted from the official quickstart. Compiled here (langgraph 1.2.22) and driven with a scripted
# stand-in for the model. Not run against a real model.
import operator
from typing import Annotated, Literal, TypedDict
from langchain.chat_models import init_chat_model
from langchain.messages import AnyMessage, ToolMessage
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph
model = init_chat_model("anthropic:claude-opus-5-5").bind_tools([check_underwriting_rules])
class State(TypedDict):
messages: Annotated[list[AnyMessage], operator.add]
def llm_call(state: State) -> dict: # the model asks
return {"messages": [model.invoke(state["messages"])]}
def tool_node(state: State) -> dict: # your node decides and calls
return {"messages": [ToolMessage(content=str(check_underwriting_rules(**call["args"])),
tool_call_id=call["id"])
for call in state["messages"][-1].tool_calls]}
def should_continue(state: State) -> Literal["tool_node", "__end__"]:
return "tool_node" if state["messages"][-1].tool_calls else END
builder = StateGraph(State)
builder.add_node("llm_call", llm_call)
builder.add_node("tool_node", tool_node)
builder.add_edge(START, "llm_call")
builder.add_conditional_edges("llm_call", should_continue, ["tool_node", END])
builder.add_edge("tool_node", "llm_call")
graph = builder.compile(checkpointer=InMemorySaver()) # development only: this saver keeps state in RAM
The fragment is three times longer than its neighbours, and that is the lesson of rung 5. The runtime does not hide the loop. It makes the loop a drawing you own, so that each step can be saved.
LlamaIndex
What it is. "The LlamaIndex framework is our original open-source toolkit for RAG and agents." Its agents are pre-built Workflows, and its documentation describes a Workflow as event-driven and built from steps.
Jobs it runs for you. Retrieval over your documents, the loop, tools from plain functions, structured output through output_cls, step retries, instrumentation.
What stays with you. State between runs: "By default, the AgentWorkflow is stateless between runs." You serialize a Context with ctx.to_dict() and store it. Surviving a crash needs the DBOS integration or your own snapshots.
# Adapted from the official documentation. Constructed here (llama-index-core 0.14.26,
# llama-index-llms-anthropic 0.13.0); the function was not called.
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.anthropic import Anthropic
agent = FunctionAgent(
tools=[check_underwriting_rules],
llm=Anthropic(model="claude-opus-5-5"),
system_prompt=INSTRUCTIONS,
)
async def triage(email_text: str) -> str:
return str(await agent.run(user_msg=email_text)) # calls the modelCrewAI
What it is. "CrewAI is the leading open-source framework for orchestrating autonomous AI agents and building complex workflows." Crews are "teams of autonomous agents". Flows are "structured, event-driven workflows that manage state and control execution".
Jobs it runs for you. The loop, prompt templates built from role, goal and backstory, memory, structured task output, checkpoints, and a pause for human feedback.
What stays with you. The model choice, if you want one: with no llm, the agent uses OPENAI_MODEL_NAME or the default the documentation names. Resuming from a checkpoint is a call you make. Hosted tracing is the vendor's AMP product.
# Adapted from the official documentation, which shows @tool as a decorator.
# Constructed here (crewai 1.15.41); the last line was not run.
from crewai import Agent
from crewai.tools import tool
agent = Agent(
role="Quote triage underwriter",
goal="Decide whether to quote, ask a question, or send the request to a person.",
backstory="You work at an insurer's intake desk and check the underwriting rules first.",
tools=[tool("check_underwriting_rules")(check_underwriting_rules)],
)
result = agent.kickoff(email_text) # calls the modelPydantic AI
What it is. "Pydantic AI is the Python AI SDK: a typed, extensible agent loop with every model a string swap away".
Jobs it runs for you. The loop, typed tools and dependencies, validated output ("If validation fails, the agent is prompted to try again"), usage limits, fallback, OpenTelemetry spans, and a test model that needs no network.
What stays with you. Storing the conversation between runs. Durability comes from one of "nine engines: Temporal, DBOS, Prefect, Restate, AWS Lambda, Airflow, Absurd, pgtask, and Azure Durable Task", which you operate.
# Adapted from the official documentation. Run here (pydantic-ai-slim 2.68.0) with the library's
# TestModel in place of a model: no network, no key.
from pydantic_ai import Agent
from pydantic_ai.models.test import TestModel
agent = Agent("anthropic:claude-opus-5-5", instructions=INSTRUCTIONS)
agent.tool_plain(check_underwriting_rules)
with agent.override(model=TestModel()):
result = agent.run_sync(email_text)
print(result.usage().requests, result.usage().tool_calls)
2 1
Two requests and one tool call: the loop ran once around the tool. TestModel invents the arguments, so the tool's answer in this run means nothing. It is the one fragment on this page whose loop was executed.
Microsoft Agent Framework
What it is. "Build AI agents and multi-agent workflows in .NET, Python, and Go with Microsoft Agent Framework." The overview calls it "the next generation of both Semantic Kernel and AutoGen".
Jobs it runs for you. The loop, tools from plain functions, three kinds of middleware (agent, function, chat), sessions, tool approval, OpenTelemetry, and graph workflows that save a checkpoint "at the end of each superstep".
What stays with you. Retry and fallback for model calls, which the pages read show only as middleware you write. Checkpoints are a workflow feature; a single agent's session survives a restart only if you serialize it.
# Adapted from the official documentation. Constructed here (agent-framework 1.26.0,
# agent-framework-anthropic 1.0.0b261112, a pre-release); the function was not called.
from agent_framework import Agent
from agent_framework.anthropic import AnthropicClient
agent = Agent(
client=AnthropicClient(model="claude-opus-5-5"),
instructions=INSTRUCTIONS,
tools=check_underwriting_rules,
)
async def triage(email_text: str) -> str:
return (await agent.run(email_text)).text # calls the modelOpenAI Agents SDK
What it is. The SDK "enables you to build agentic AI apps in a lightweight, easy-to-use package with very few abstractions".
Jobs it runs for you. The loop with a turn limit, tools, handoffs between agents, guardrails, sessions, approval with a serializable RunState, and tracing.
What stays with you. Retries: "The SDK does not retry general model requests unless you set ModelSettings(retry=...)". Crash recovery, through one of four named partners. And a decision about tracing, which "is enabled by default" and "exports the spans and traces to the OpenAI backend".
# Adapted from the official quickstart. Constructed here (openai-agents 0.24.0); the last line was not run.
# No model is named, so the SDK's default OpenAI model applies.
from agents import Agent, Runner
from agents.decorators import tool
agent = Agent(
name="Quote triage",
instructions=INSTRUCTIONS,
tools=[tool(check_underwriting_rules)],
)
result = Runner.run_sync(agent, email_text) # calls the modelGoogle ADK
What it is. ADK is "the open-source agent development framework that lets you build, debug, and deploy reliable AI agents".
Jobs it runs for you. The loop, tools from plain functions ("automatically wrapped as FunctionTools"), graph workflows, sessions, tool confirmation, context compaction, evaluation with adk eval.
What stays with you. A session store that survives a restart: with InMemorySessionService, the documentation warns that conversation data "is lost if the application restarts". Check feature combinations, too. The tool confirmation page lists DatabaseSessionService as "not supported by this feature".
# Adapted from the official Python quickstart, which uses a Gemini model.
# Constructed here (google-adk 2.14.1); not run. The quickstart runs an agent with: adk run <folder>
from google.adk.agents.llm_agent import Agent
root_agent = Agent(
model="gemini-flash-latest",
name="quote_triage",
description="Triages homeowners quote emails.",
instruction=INSTRUCTIONS,
tools=[check_underwriting_rules],
)Strands Agents
What it is. "Strands Agents is an open-source SDK for building and running AI agents in Python and TypeScript." Its README describes "a model-driven approach to building AI agents in just a few lines of code".
Jobs it runs for you. The loop, tools, conversation management (a sliding window by default), session managers, an approval step, retries on throttling, traces.
What stays with you. Limits: "No limit applies unless you pass one." Amazon Bedrock is the default model provider, so another provider is a line you add. The pages read say nothing on resuming a run after a crash.
# Adapted from the official documentation. Constructed here (strands-agents 1.65.0); the last line was not run.
from strands import Agent, tool
from strands.models.anthropic import AnthropicModel
agent = Agent(
model=AnthropicModel(model_id="claude-opus-5-5", max_tokens=4096),
tools=[tool(check_underwriting_rules)],
system_prompt=INSTRUCTIONS,
)
result = agent(email_text) # calls the modelTemporal
What it is. "Temporal is a scalable platform that ensures the Durable Execution of application code". It is not an agent framework. It runs any code durably, and an agent loop is one kind of code.
Jobs it runs for you. Jobs 3 and 4 for any code. A Workflow's steps are recorded in an Event History, and after a crash Temporal "replays the Event History step by step" to rebuild the state. Activities are retried, with unlimited attempts by default.
What stays with you. Every other job. The loop is Workflow code you write, or a framework's loop placed inside one: Temporal's Python page lists integrations with the OpenAI Agents SDK, Pydantic AI, Google ADK and LangGraph, among others. Workflow code must be deterministic, so each model call goes into an Activity.
There is no fragment here. An agent with one tool on Temporal is a Workflow, an Activity and a Worker, and the documentation offers no short form of it that matches the other tabs.
Read the nine fragments side by side. The tool function did not change once. Its name, its type hints and its docstring were enough in every fragment. What changed is the vocabulary around it: system_prompt, instructions, instruction, and in one case role, goal and backstory.
6. What the framework costs, and when each cost matters#
None of the following is a reason to avoid a framework. Each is a cost with a condition, and the condition decides whether you pay it.
A second vocabulary. The same wrapping point is called middleware in LangChain and Microsoft Agent Framework, a capability in Pydantic AI and an intervention in Strands, and the OpenAI Agents SDK has tool guardrails that run before and after a tool. Under each name is the model call from One call. This cost matters when the team does not know the bottom layer. A team that has read harness.py translates the vocabulary in an hour. A team that started on rung 4 has only the vocabulary.
Upgrades. A framework changes faster than your business rules. Pages read for this chapter in one day showed a Temporal wrapper class marked for removal in the next major version of Pydantic AI, a set of ADK workflow agents marked deprecated in favour of graphs, CrewAI's code-execution settings deprecated, and a LangGraph default changed between two patch releases. The fact baseline for this site, checked two days earlier, was already out of date for at least two rows. This cost matters in proportion to how much of the framework's surface your code touches.
Defaults you did not choose. The miniature stops after MAX_TURNS = 8 because the team picked 8. In this example a normal quote takes five turns. On rung 4 the number is someone else's.
| Product | A default you inherit | Source |
|---|---|---|
| LangChain | The limits of ModelCallLimitMiddleware: "Defaults to no limit." For an agent without it, the pages read state no limit. | Built-in middleware page |
| LangGraph | recursion_limit is 10,007 super-steps (10,000 before version 1.1.4) | Graph API page |
| CrewAI | max_iter: the attribute table says 20; version 1.15.41 as installed reports 25 | Agents page; the installed package |
| Pydantic AI | request_limit is 50 | Retries page; the installed package |
| OpenAI Agents SDK | max_turns is 10 in version 0.24.0. Tracing is on. | The installed package; tracing page |
| Google ADK | max_llm_calls is 500 | RunConfig page; the installed package |
| Strands Agents | No limit unless you pass one. Six attempts on throttling. | Agent loop and retry pages |
| Microsoft Agent Framework | max_function_calls is None, "which means unlimited" | Function tools page |
A loop that goes wrong on a quote email is stopped after 8 turns by the miniature. Under these defaults it is stopped after 10, 20 or 25, 50, 500 or 10,007 of whatever unit each product counts, or not at all. Other defaults add text or send data: CrewAI summarizes the conversation at the context limit unless respect_context_window is off, and sends usage telemetry unless CREWAI_DISABLE_TELEMETRY is set. This cost matters most where a default touches money or data, and it is paid by teams that never read the defaults.
A hidden loop is harder to debug. When a quote comes back wrong, the first question is what the model received. In the miniature that is harness.py --dry-run. Behind a framework the request is assembled by its loop and your middleware, so the answer is in a trace. Tracing stops being optional on rung 4, which is why column 8 of the table deserves as much attention as column 3.
Lock-in sits at the state layer. Swapping one constructor for another is a small change, as the tabs show. Saved state is different. A checkpoint is written in the runtime's own format, keyed by its own ids, and read back by its own code. A run that is paused at the approval gate today must be resumable after next month's upgrade, by the new version, from the old row. This cost matters only for runs that live long. A loop that finishes in ten seconds has no state to be locked into. A quote that waits two days for an underwriter does.
7. How to read any new framework in ten minutes#
A new framework appears every few weeks, and each can be read the same way. Open its documentation and ask one question per job. Write down the answer, or "the docs do not say".
- Context. What does it add to my prompt that I did not write, and can I print the final request?
- Tools. How does a function become a tool, and in which process does the tool run?
- Control flow. What stops the loop if the model never stops asking, and what is that default?
- State. What survives a restart with no configuration? What survives a crash in the middle of a run, and whose database holds it?
- Identity and approval. Can a run pause for a person? Where do I check who the caller is? Expect the second answer to be "in your code".
- Output. How do I get a typed result, and what happens when validation fails?
- Reliability and cost. Are retries on or off by default? Is there a budget in tokens or money, or only a count of calls?
- Observability. Where do traces go by default, and can they go to a store I choose?
Eight answers place the product in the table of section 4. The engineer's question from the opening now has an answer too. A framework would have written the loop, the schemas, the retry, the parser and the trace events: the middle column of section 1. The team would still have written the rules, the roles, the approval policy, the outbox and the golden set. The decision between the two was never about line count. It is about whether the generic half is worth owning, and that depends on how often the team needs to look inside it.
How each fragment and fact on this page was checkedVendor pages read on 2026-11-16; nine agents constructed, none called a model
| Item | Status |
|---|---|
| Miniature commands in section 3 | Run for this page with --fake. Real output; token counts come from the scripted model. |
| Three-node LangGraph graph in section 3 | Run for this page with InMemorySaver and no model. Real output. |
| Pydantic AI fragment | Run with the library's TestModel. Real output. |
| LangGraph fragment in its tab | Compiled, then driven once with a scripted stand-in for the model. |
| The other seven fragments | The agent object was constructed with a placeholder key. The marked line was not run. |
| Quoted sentences | Read on the vendor's page on 2026-11-16. Sources per claim are in the fact log for this page. |
| Cells marked "not re-read today" or "not confirmed today" | Stated in the baseline of 2026-11-14 and not found again on the pages read today. |
The fragments leave out error handling, streaming, reading results and every guard the miniature has. apply_rules and email_text stand for code the reader supplies. The scratch environment used Python 3.12 and the package versions named in each fragment.
What comes next#
The team can now say what rungs 4 and 5 would write for it, and what they would hand back. On both rungs the loop still runs in a process the team operates, the checkpoints sit in a database the team runs, and the tools execute on the team's machines. That is the last thing left to give away. Managed platforms looks at products that take the servers, the sandbox and the session store, and at what happens to the right-hand column when the left-hand column runs in someone else's data centre.