All chapters
  1. Start hereAn agent is a model call in a loop
  2. Chapter 1One call: what a model is from the outside
  3. Chapter 2The loop: where a call becomes an agent
  4. Chapter 3The abstraction ladder: six rungs over one call
  5. Chapter 4The eight jobs of a harness
  6. Chapter 5Context: deciding what the model sees
  7. Chapter 6Tools: what the model may ask for, and where it runs
  8. Chapter 7MCP: one way to plug tools into any harness
  9. Chapter 8RAG: retrieval is context assembly
  10. Chapter 9Control flow and state: stopping, pausing and resuming
  11. Chapter 10Guardrails: identity, approval and the output contract
  12. Chapter 11Reliability and cost: retries, budgets and the failures that will happen
  13. Chapter 12Observability and evals: knowing what the agent did, and whether it was right
  14. Chapter 13Many agents: more loops, and what each one costs
  15. Adopt · 1SDKs: the same loop in Python, TypeScript and Java
  16. Adopt · 2Coding harnesses: a finished harness for software work
  17. Adopt · 3Frameworks and graph runtimes: the loop as a library
  18. Adopt · 4Managed platforms: someone else runs the loop
  19. Adopt · 5Build your own: a whole harness in one file
  20. DecideBuild or adopt: who runs each job?
  21. ReferenceGlossary
  22. AboutAbout this site
Decide

Build or adopt: who runs each job?

The ecosystem on one map: for every product and every job, whether it is built in, integrated, managed or left to you. And how to decide, job by job.

The design review from The ladder meets again. Then, the team had a thirty-line script and three proposals it could not compare. Now the script is harness.py, about 450 lines, and the proposals are back in a new form. One engineer wants to keep the hand-written harness on the provider SDK. One wants to move it onto an agent framework, "because it has checkpoints and approval built in". One wants a managed platform, "so that we stop running servers for this".

Last time each proposal came with a dozen lines of code, and the room compared syntax. This time nobody shows code. The first question comes from the engineer who wrote the approval gate: "On your proposal, who runs job 5?" The framework's advocate answers at once: the pause is built in, and the check of who the caller is stays in our code. The platform's advocate needs a minute, and then says the same thing in other words.

That exchange is new. The team can now read a proposal as eight answers, one per job, and two proposals that looked different turn out to agree on the job that matters most to an insurer. What the team still lacks is the whole picture: every product it has heard named, on one sheet, read the same way. And then a way to choose that does not end in "which one is best".

You do not choose a product for an agent. You choose an owner for each of eight jobs. A product is a bundle of such choices that someone else made, and the decision is which bundle leaves the right jobs in your hands.

1. Why the products cannot be sorted into boxes#

The obvious way to draw the market is a set of boxes: SDKs here, frameworks there, platforms on top. It fails on the first products you try.

LangChain's documentation calls create_agent an agent harness, and that loop runs on LangGraph, which is a separate product on another rung. The OpenAI Agents SDK and Strands Agents are SDKs by name, and both ship sessions, approval and tracing, which is framework work. Amazon Bedrock AgentCore is a managed platform whose harness module is, according to its documentation, powered by Strands Agents. LangSmith is a tracing product that also deploys agents. A box with a wall around it cannot hold any of these.

So this site does not sort products. It asks each one the question from The eight jobs: who runs this job? A product gets one row and eight answers. Each row also names a primary role, the thing the product is mainly for, only so that the rows can be grouped into tables of a readable size. The role is a label for the table. The cells are the content.

Each cell uses one of five words, with a few words of detail in the vendor's own terms.

Word in the cellWhat it meansWhat it costs you
YOU BUILDThe documentation leaves the job to your codeYour engineering time, now and at every change
BUILT INThe product ships the job as a library or a feature. You still configure and operate itIts vocabulary, its defaults and its upgrades
INTEGRATESThe product hands the job to a partner product, which the cell namesA second product to run, and the seam between the two
MANAGEDThe vendor runs the job as a serviceControl over where it runs, how it fails and what it stores
NOT ITS JOBThe product does not aim at this job. A fact about its scope, not a faultNothing, as long as you know another owner is needed

There is a sixth entry: "the docs do not say". It means the vendor pages read for this site did not answer. It never means the feature is absent.

The diagram shows the idea on one job, job 4: saving a run so it can continue. Four different owners can hold it. One part stays with the insurer under all four.

Diagram: One job, four possible owners · and the part that stays with you

Read left to right. The job on the left is job 4 from the quote example: save the run, so it can continue after the underwriter answers. The middle column shows the four owners a product map can name. Whichever one you pick, the box on the right is still yours. Click any box.

  1. 1 · A job that must be done
    1. Save the run, continue it later: Job 4. The reply waits for a person, perhaps for two days. The run must be saved, the process must be free to exit, and another process must continue it.
  2. 2 · Who can run it
    1. You build it: The job is your code and your database. You can read every line, and you maintain every line.
    2. Built in: a library does it: The product ships the job. You still pass it a database, operate it, and follow its upgrades.
    3. Integrates: a named partner does it: The product hands the job to a second product. You now run two, and the seam between them.
    4. Managed: a vendor runs it: The vendor stores the session as a service. You give up control over where the state lives and what happens in an outage.
  3. 3 · Yours under every owner
    1. Make a repeated step harmless: Every owner in the middle column can run a step again after a crash or a resume. Only your code knows that the reply email must not go out twice.

Three limits apply to every table below, and they belong before the tables, not after. The cells were read on vendor documentation on 2026-11-16, or come from this site's checked baseline of 2026-11-14 where a row says so. A cell is a reading of documentation, not a test: nothing here was installed and measured for this page. And the set of products is a selection of the ones this series discusses. It is not a census of the market, and the order of rows is not a ranking.

2. The ecosystem map, in five tables#

The ecosystem map is the full table of products by jobs. It is wide and long, so it is split by primary role. Each product page of this site shows only its own rows, in more detail and with the source of every cell. Where a table here shortens a cell, the product page has the full wording.

Provider SDKs and agent SDKs#

These are rungs 2 and 3: a library that wraps the call, and a library that wraps the loop. The first three rows are from SDKs. The next two carry "SDK" in their names and are read in detail on Frameworks. The last row is a model API that can keep state on the server, taken from the baseline.

Product and primary role1 Context2 Tools3 Control flow4 State, durability5 Identity, approval6 Output7 Reliability, cost8 Observability, evals
Anthropic client SDKs · provider SDKYOU BUILD: you send every message on every callYOU BUILD: the schema and the function, in your processYOU BUILD: the loop and its stopYOU BUILDYOU BUILDBUILT IN: structured outputs with a JSON SchemaBUILT IN for one request: two retries, a time limit, token counts. A budget across turns: YOU BUILDBUILT IN: logging; one trace span per request on the Python page. Evals: YOU BUILD
The tool runner (beta) · a helper inside the provider SDKBUILT IN: the history of the run, optional compactionBUILT IN: derives the schema and calls your function, in your processBUILT IN: the loop. A turn limit only if you set max_iterationsYOU BUILDYOU BUILDThe docs do not sayAs the provider SDK. A budget: YOU BUILDThe docs do not say
Claude Agent SDK · agent SDKBUILT IN: automatic compaction, skills and memory filesBUILT IN: built-in tools, MCP, in-process custom tools, a sandbox option. Isolation is your deploymentBUILT IN: the loop, max_turns, subagents, hooksBUILT IN: sessions on local disk, resume and fork, an adapter for external storageBUILT IN: permission modes, allow and deny rules, a callback, hooks. The end user's identity: the docs do not sayBUILT IN: output_format with a JSON SchemaBUILT IN: max_budget_usd on an estimate, fallback_model. Side effects that must happen once: the docs do not sayBUILT IN: OpenTelemetry export when you switch it on. Evals: the docs do not say
OpenAI Agents SDK · agent SDKBUILT IN: instructions, sessions, an input filterBUILT IN: function tools, hosted tools, MCP, sandbox agentsBUILT IN: the loop, handoffsBUILT IN: sessions on several stores, RunState. INTEGRATES: Dapr, Temporal, Restate, DBOSBUILT IN: guardrails, needs_approval. Authorization: YOU BUILD, inside the toolBUILT IN: output_typeBUILT IN: max_turns, usage tracking, opt-in retries. Fallback: the pages read do not sayBUILT IN: tracing, on by default. Evals: not in the pages read
Strands Agents · agent SDKBUILT IN: conversation managersBUILT IN: tools, MCP, Docker and SSH sandboxesBUILT IN: the loop, optional limitsBUILT IN: session managers for files and S3. Durable execution: the pages read do not sayBUILT IN: HumanInTheLoop, CedarAuthorizationBUILT IN: structured_output_modelBUILT IN: retries on throttling, limits you passBUILT IN: OpenTelemetry traces
OpenAI Responses and Conversations API · model API with optional stored stateMANAGED, optional: server-side compactionMANAGED: built-in tools, remote MCPThe docs do not say who runs the loopMANAGED, optional: responses stored 30 days by default; conversation objects persistNOT ITS JOB for your own tools. Approval settings exist for remote MCP toolsBUILT IN: structured outputsBUILT IN: prompt cachingNot confirmed on a page read

The first two rows say YOU BUILD in columns 4 and 5, and so does every provider SDK. A library that wraps a request has no place to keep a run and no way to know your users. The step from the second row to the third is the largest on the map for a single install: the Agent SDK fills six columns that the tool runner leaves empty. Its price is on SDKs: a long-lived process per session, in Python or TypeScript only.

Coding harnesses#

A coding harness is a finished harness for software work, so every job is already built. The cells below are feature names from the translation table on Coding harnesses, read here as BUILT IN. That page has the files, the flags and the defaults.

Product and primary role1 Context2 Tools3 Control flow4 State, durability5 Identity, approval6 Output7 Reliability, cost8 Observability, evals
Claude Code · coding harnessBUILT IN: CLAUDE.md, automatic compaction, skillsBUILT IN: tools, MCP. A sandbox for shell commands, off by defaultBUILT IN: plan mode, subagents, claude -p; --max-turns in scripted runsBUILT IN: --continue, --resume, /rewindBUILT IN: six permission modes; allow, ask and deny rules; hooksBUILT IN: --json-schemaBUILT IN: /usage, --max-budget-usdBUILT IN: JSONL transcripts, OpenTelemetry export
Codex CLI · coding harnessBUILT IN: AGENTS.md, skills. Compaction: two hook events; beyond that the docs do not sayBUILT IN: tools, MCP, sandbox_mode with three levelsBUILT IN: subagents, codex exec. Plan mode and turn limits: the docs do not sayBUILT IN: codex resumeBUILT IN: approval_policy, hooksBUILT IN: --output-schemaToken counts in the turn.completed eventBUILT IN: an event stream with --json. Tracing: the docs do not say
GitHub Copilot CLI · coding harnessBUILT IN: instruction files, automatic compaction, skillsBUILT IN: tools, MCP, /sandbox enable, trusted directoriesBUILT IN: plan mode, autopilot mode, subagents, copilot -pBUILT IN: --continue, --resumeBUILT IN: --allow-tool, --deny-tool, hooksThe docs do not sayBUILT IN: /usage, /contextBUILT IN: session state and logs on disk. Tracing: the docs do not say
Antigravity CLI · coding harnessBUILT IN: rules files, automatic compaction, skillsBUILT IN: tools, MCP, --sandbox, off by defaultBUILT IN: --mode plan, subagents, agy -p. Turn limit: the docs do not sayBUILT IN: --continue, /resume, /rewindBUILT IN: three modes; allow, ask and deny rules; hooksBUILT IN: --json-schemaBUILT IN: /usage, /creditsBUILT IN: transcript.jsonl. Tracing: the docs do not say
Cursor and Cursor CLI · coding harnessBUILT IN: rules files, skills. Compaction: one hook event; beyond that the docs do not sayBUILT IN: tools, MCP, --sandbox enabled or disabledBUILT IN: plan mode, subagents, agent -p. Turn limit: the docs do not sayBUILT IN: agent resume, agent lsBUILT IN: permissions.allow and deny, hooksThe docs do not sayThe docs do not sayBUILT IN: a transcript file. Tracing: not read
Grok Build · coding harnessBUILT IN: AGENTS.md, automatic compaction, skillsBUILT IN: tools, MCP, --sandbox with a profile, off by defaultBUILT IN: /plan, subagents, grok -p. Turn limit: the docs do not sayBUILT IN: --continue, --resumeBUILT IN: three modes, [permission] rules, hooksThe docs do not sayBUILT IN: /usageBUILT IN: /transcript, /export. Tracing: the docs do not say

Columns 1 to 5 are full in every row, and columns 6 to 8 are where "the docs do not say" collects. That is the order in which a developer at a keyboard needs the jobs. A production agent needs them in the reverse order. Column 5 is also the one to read twice: the gate in every row asks the developer who is present, and that is a different person from the underwriter who approves a quote.

Agent frameworks and graph runtimes#

These are rungs 4 and 5: the loop as a library, and a runtime that saves the state as the run proceeds. The rows are from Frameworks, shortened. That page also lists the defaults each product sets for you.

Product and primary role1 Context2 Tools3 Control flow4 State, durability5 Identity, approval6 Output7 Reliability, cost8 Observability, evals
LangChain create_agent · agent frameworkBUILT IN: system_prompt, middleware. Summarization only if you set a triggerBUILT IN: tools from functions. The shell tool runs on the host unless you pick DockerExecutionPolicyBUILT IN: the loop, on LangGraphBUILT IN once you pass a checkpointerBUILT IN: HumanInTheLoopMiddleware. Caller identity: the pages read do not sayBUILT IN: response_formatBUILT IN: retry, fallback and call-limit middlewareINTEGRATES: LangSmith
LangGraph · graph runtimeBUILT IN: state, store. Prompts are yoursBUILT IN: a prebuilt ToolNode class. The quickstart writes the tool node by hand. Sandbox: the pages read do not sayBUILT IN: the graphBUILT IN: checkpointers for memory, SQLite, PostgreSQL, MongoDB, Cosmos DBBUILT IN: interrupt. Identity: the pages read do not sayThe pages read do not sayBUILT IN: RetryPolicy, node timeouts. Token budget: the pages read do not sayINTEGRATES: LangSmith
LlamaIndex · agent framework, data-centricBUILT IN: retrieval over your data, memory classesBUILT IN: tools from functions. Sandbox: not re-read todayBUILT IN: event-driven WorkflowsYOU BUILD the saving, with your own store. INTEGRATES: DBOS, to survive a crashBUILT IN: events that wait for a person. Identity: the pages read do not sayBUILT IN: output_clsBUILT IN: retry_policy on steps. Agent turn limit: the pages read do not sayBUILT IN: instrumentation, evaluation modules. INTEGRATES: Arize Phoenix, Langfuse, Opik
CrewAI · agent frameworkBUILT IN: role templates, memory. Summarizes at the context limit by defaultBUILT IN: tools, MCP. Code execution is deprecatedBUILT IN: Crews and FlowsBUILT IN: checkpoints to JSON files or SQLite; @persist on Flows. Resuming is a call you makeBUILT IN: human_input, @human_feedback. Identity: YOU BUILDBUILT IN: output_pydantic, output_jsonBUILT IN: max_iter, max_retry_limit, max_rpm. Fallback: the pages read do not sayMANAGED: CrewAI AMP tracing. INTEGRATES: Langfuse, Arize Phoenix and others
Pydantic AI · agent framework, typedBUILT IN: instructions, typed dependenciesBUILT IN: tools, MCP. Sandbox: not confirmed todayBUILT IN: the loop, pydantic-graphYOU BUILD: you store the message history. INTEGRATES: nine durable execution enginesBUILT IN: requires_approval. Identity: YOU BUILDBUILT IN: output_type, with a validation retryBUILT IN: UsageLimits, ModelFallback, retriesBUILT IN: OpenTelemetry, Pydantic Evals. INTEGRATES: Logfire
Microsoft Agent Framework · agent frameworkBUILT IN: context providers. Compaction is experimental in PythonBUILT IN: tools, MCP. Shell tools: a beta packageBUILT IN: agents and graph workflowsBUILT IN: AgentSession, which you serialize; workflow checkpoints to memory, file or Cosmos DBBUILT IN: approval_mode. Identity: YOU BUILDBUILT IN: response_formatYOU BUILD: retry and fallback as middleware. Tool-call limits default to unlimitedBUILT IN: OpenTelemetry. Evals: the pages read do not say
Google ADK · agent frameworkBUILT IN: instruction templates, compaction. Context caching is experimentalBUILT IN: tools, MCP, OpenAPIBUILT IN: graph workflowsBUILT IN: session services for memory, a database or Agent Platform; resumable workflowsBUILT IN: require_confirmation, tool authenticationBUILT IN: output_schemaBUILT IN: max_llm_calls, retry on graph nodes. Python fallback: the pages read do not sayBUILT IN: logging, OpenTelemetry traces, adk eval

Column 3 says BUILT IN in every row, which is what a framework is. Column 5 splits inside its own name: the pause for a person is built in everywhere, and the check of who the caller is comes back to your code in almost every row. Column 4 hides two promises under one heading. "The next message remembers the last one" is a session store. "The run survives a crash in the middle" is durable execution, and three rows answer that one with YOU BUILD or a partner.

Managed platforms#

This is rung 6: the vendor runs the loop, the sandbox and the stored session. The rows are from Managed platforms. They describe one mode of two products: the harness module of AgentCore, and prompt agents on Foundry.

Product and primary role1 Context2 Tools3 Control flow4 State, durability5 Identity, approval6 Output7 Reliability, cost8 Observability, evals
Claude Managed Agents (beta) · managed harnessMANAGED: prompt caching, compactionMANAGED: a container per session, or a worker you host. Custom tools: YOU BUILDMANAGEDMANAGED: session history, sandbox checkpointMANAGED: permission policies, vaultsThe docs do not sayMANAGED: session budget, automatic retry of transient errorsMANAGED: session viewer, usage, OpenTelemetry tracing
OpenAI Agents API (beta) · managed harnessMANAGED: context compactionMANAGED: a hosted sandbox, or an executor you host. Function tools: YOU BUILDMANAGEDMANAGED: session stateMANAGED: vaults, automatic reviewThe docs do not sayMANAGED: sessions resume. Retrying a failed request: YOU BUILDMANAGED: tracing on by default, export
Amazon Bedrock AgentCore, harness · managed harnessMANAGED: truncation by sliding window or summarizationMANAGED: a microVM per session, Gateway, Code Interpreter, Browser. Inline functions: YOU BUILDMANAGED: the harness. On Runtime alone, the loop is your codeMANAGED: Memory, short and long termMANAGED: Identity, PolicyThe docs do not sayMANAGED: limits on iterations, time and tokensMANAGED: Observability, Evaluations
Foundry Agent Service, prompt agents · managed harnessMANAGED: input truncated to the model's context sizeMANAGED: Code Interpreter sandbox, Toolboxes, MCP. Function calling: YOU BUILDMANAGED. A hosted agent runs your code insteadMANAGED: Microsoft's storage, or your own Azure resourcesMANAGED: Entra agent identity, role-based access, content filters, approval for MCP toolsThe docs do not sayYOU BUILD: retry rate-limit errors with backoffMANAGED: tracing, metrics, evaluations
Google Agent Runtime · managed hosting for your agent codeThe docs do not sayMANAGED: runs the container you deploy. Code Execution sandboxThe docs do not sayMANAGED: Sessions, Memory BankMANAGED: Agent Identity, Agent GatewayThe docs do not sayThe docs do not sayMANAGED: tracing, evaluation

Column 2 has the same shape in four rows: a managed sandbox, then YOU BUILD for your own tools. A vendor can run a shell. It cannot run the insurer's underwriting rules. Column 6 is empty in all five rows: none of the agent definitions read has a field for an output schema, so the check of the final answer stays in your application. And the word "managed" covers two offers: a vendor that runs its harness for you, and a vendor that hosts a loop you wrote with a framework. The last row is the second kind.

Products that take one job#

Some products do not run the loop at all. They sit beside it and take one job, or one and a half. A row of NOT ITS JOB is not a weak product. It is a narrow one, and it combines with any rung. The Temporal row is from Frameworks and Managed platforms. The other four rows come from the baseline, and each product's front page was read again on 2026-11-16.

Product and primary role1 Context2 Tools3 Control flow4 State, durability5 Identity, approval6 Output7 Reliability, cost8 Observability, evals
Temporal · durable execution platformNOT ITS JOBNOT ITS JOB: a tool call is one of your ActivitiesBUILT IN: Workflows as codeBUILT IN: Event History and replay. Temporal Cloud: MANAGED storage; your Workers stay with youNOT ITS JOB. A Signal lets a Workflow wait for a personNOT ITS JOBBUILT IN: Activity retries, unlimited attempts by defaultBUILT IN: metrics, tracing, logging
LangSmith, with Agent Server · tracing and eval platform that also deploysNOT ITS JOB at run time. Prompt engineering is a named product areaNOT ITS JOBNOT ITS JOB: the graph is your codeMANAGED: Agent Server, with persistence and a task queue. Cloud or hybridAgent Server: custom authentication and authorization handlers, per the baselineNOT ITS JOBAn LLM Gateway is named. Details: the docs do not sayBUILT IN: tracing, dashboards, alerts, offline and online evaluation
Langfuse · tracing and eval platform, open sourceBUILT IN in part: prompt managementNOT ITS JOBNOT ITS JOBNOT ITS JOBNOT ITS JOBNOT ITS JOBBUILT IN: an AI Gateway that tracks spend per user, key and modelBUILT IN: tracing, evaluation, datasets
Arize Phoenix · tracing and eval tool, open sourceBUILT IN in part: prompt management, a playgroundNOT ITS JOBNOT ITS JOBNOT ITS JOBNOT ITS JOBNOT ITS JOBNOT ITS JOBBUILT IN: tracing, evaluation, datasets and experiments
LiteLLM · model gatewayNOT ITS JOBNOT ITS JOB. Its README also names an MCP gatewayNOT ITS JOBNOT ITS JOBBUILT IN: virtual keys, guardrailsNOT ITS JOBBUILT IN: retries, fallbacks, cooldowns, timeouts, load balancing, spend trackingINTEGRATES: logging to other tools

Three jobs can be bought alone: job 4 from a durable execution service, job 7 from a gateway, job 8 from a tracing product. No row on this table takes job 2 or job 6. The two filled cells in column 5 are about keys for model calls and access to a deployment server. Neither is about who may approve a quote. The NOT ITS JOB cells in the four baseline rows are this site's reading of each product's scope. No vendor page states them.

Durable execution has more names than one row. The cells above name DBOS, Restate, Dapr and others as partners of frameworks. DBOS describes itself as "a library for building reliable programs", and Restate as "a lightweight runtime to turn AI agents, workflows, and backend services into durable processes". Neither was read beyond its front page, so neither has a row.

Protocols: the connectors between the rows#

Three names come up next to these products and are not products. Nobody runs a job by adopting a protocol. A protocol decides how two owners of jobs talk to each other, which matters most on the day one of them is replaced.

ProtocolWhat it connectsThe job it touchesStatus as read
MCP, the Model Context ProtocolA harness to tool serversJob 2. Home: MCPSpecification version 2026-07-28
A2A, the Agent2Agent protocolOne agent system to another that it cannot see insideJobs 2 and 3, across a company boundary. See Many agentsVersion 1.0, under the Linux Foundation
OpenTelemetry, GenAI semantic conventionsA harness to any trace storeJob 8. See Observability and evalsMarked "Development", not stable
How the map was built, and which names changedThree sources, one rule for conflicts, five renames

Each cell has one of three sources. Rows that a product page of this site covers were copied from that page, which read the vendor's documentation on 2026-11-16. Rows that no product page covers come from the baseline of 2026-11-14, and the vendor's front page or the page behind the cell was read again on 2026-11-16. A cell that the baseline marks as unknown is written as "the docs do not say".

Where a product page and the baseline differ, the product page wins, because it is the later reading. Three such cells are on the map. LlamaIndex, job 4: the baseline said built in, and the page read on 2026-11-16 says "There is no built-in checkpointer to enable." Foundry Agent Service, job 6: the baseline said managed, and the schema setting was found only in a workflow designer in preview. Pydantic AI, job 4: the baseline named three partner engines, and the overview now names nine. Two days were enough to move three cells, which is the best argument for reading a cell as dated.

One baseline cell is left out: a line about turn limits and token budgets for Strands Agents was not confirmed on the vendor's page, so the row shows what the product page read.

Names in this field change, and an old name in a proposal is a reason to check the date of the proposal. From the baseline: AutoGen's README says it is archived and no longer maintained, and names Microsoft Agent Framework as the successor. Semantic Kernel's README says it "is now Microsoft Agent Framework". The OpenAI Assistants API was shut down on 2026-08-26, and the Responses and Conversations APIs replace it. Vertex AI Agent Engine became Agent Runtime on 2026-04-22. Antigravity CLI replaced Gemini CLI for free-tier and Google AI Pro and Ultra users on 2026-06-18.

The fact log for this page lists every cell that does not come from a product page, with its address and the date read.

3. The same map, read by column#

Read by row, the map describes products. Read by column, it describes the jobs, and that reading is the one a decision needs. For each job: who usually runs it, and what stays with you in every row.

Job 1, context assembly. Almost every product that runs a loop also builds the request: BUILT IN from rung 3 up, MANAGED on rung 6. What they build is the mechanism: history, compaction, caching. Which data may enter the context is not in any cell. In the miniature that decision is assemble_context and may_read, and it stays where the knowledge of the insurer's data is.

Job 2, tools and where they run. The map splits this job in two. General tools and a sandbox for them are widely BUILT IN or MANAGED. Your own tools are YOU BUILD in every row that mentions them, on every rung, because their logic and data are your business. The model asks, the harness decides and calls, the tool does: a product can supply the middle actor, never the last.

Job 3, control flow. This is the most adopted job on the map. Every agent SDK, framework and managed harness runs the loop. What stays is the number: the turn limit that fits your task. Frameworks lists defaults from 10 to 10,007 and several of "no limit". The loop is theirs, and the stop is still your decision.

Job 4, state, memory and durability. Every answer appears in this column: YOU BUILD, BUILT IN with your database, INTEGRATES with a named engine, MANAGED. It is the column where products differ most, so it is the one to read first when a run must wait for a person. One thing stays in every row: making a repeated step harmless. Each vendor that resumes work says so in its own words, and in the miniature that is send_once.

Job 5, identity, permissions and approval. The pause for a person is BUILT IN or MANAGED almost everywhere. The rest of the job is not. Who the caller is, which role may run which tool, who may approve and where that is recorded: the map shows YOU BUILD or silence for these in most framework rows, and policy mechanisms without the policy in the managed rows.

Job 6, the output contract. Checking the shape of the answer is BUILT IN in most rows from the provider SDK upward, and the documentation of all five managed platforms is silent on it. Checking the meaning is in no cell at all. A quote in the standard band for a house in a flood zone passes every schema.

Job 7, reliability and cost. Retrying one request is BUILT IN at rung 2. Limits on calls are common above it. A limit in money is rare: a few cells name a budget, and one of those says it is an estimate. The bill is yours in every row, and so is the record that a side effect happened once.

Job 8, observability and evals. Traces are the easiest job to hand over: nearly every row offers one, and three products on the single-job table are built around it. Evals are offered as machinery: a runner, a scorer, a dataset store. The cases and the definition of "correct" are in no product, because they are knowledge about your business.

Put the eight "what stays" sentences side by side and one list appears. It is the same list Managed platforms found at the top of the ladder: the logic of your tools, the rule about what the agent may do, the person who approves, the judgment of results, and the bill. No rung removes any of the five.

4. How to decide, job by job#

The map does not say what to choose. It says what each choice hands to whom. The choice needs conditions, and the conditions come from the team's own situation, so start there.

Take the framework proposal from the opening. Its advocate named a real problem: the reply must wait for an underwriter, perhaps for two days, and the team wrote checkpoint, load_checkpoint and send_once to make that work. Is that a reason to adopt? It depends on three things the proposal did not state. How many such waits will there be: one gate, or many branches with long pauses? What language is the service written in, and does the product exist for it? And how much of the saved state would then be in the product's format? A run that is paused today must be resumable after next month's upgrade.

Now the managed proposal. Its advocate also named a real problem: someone operates the service, the database and the sandbox. The questions here are different. May the broker's email and every tool result be stored at a vendor? What does a session do while the vendor is unavailable? And how much of the service would be left anyway, since all three business tools run in the insurer's application on every platform?

Neither set of questions has a general answer, which is why this page has no ranking. Each direction has a cost, and the cost is of a different kind.

DirectionWhat you gainWhat you payWhen the cost is highest
Build the jobCode you can read, test and change; the request is a variable you can printWriting it, and then maintaining it at every model and API changeWhen the job is the same in every company, so your version has no advantage
Adopt a library (BUILT IN, INTEGRATES)The repeated half of a harness, written and tested by othersA second vocabulary, defaults you did not choose, upgrades on someone else's schedule, saved state in its formatWhen your code touches much of its surface, or runs live long
Adopt a service (MANAGED)No process, sandbox or session store to operateLess control over where state lives, what the network allows and what happens in an outage; a dependency for every runWhen the data is regulated, or the agent mostly uses your own tools

Three rules turn those costs into a method.

The first is the rule from The ladder, made exact: start at the lowest rung that removes a real problem you have. "We keep rewriting the retry code" is a problem, and rung 2 removes it. "A person must approve, and may take days" is a problem, and a graph runtime or a durable execution service removes it. "We might need it later" is not a problem yet, and climbing for it buys the cost without the gain.

The second: decide per job, not per system. Nothing on the map forces one owner for all eight. A provider SDK for the call, a durable execution service for the long wait, a tracing product for job 8 and your own code for identity is an ordinary combination. The single-job table exists because of it.

The third: a job that names your business stays with you, whatever the cell says. BUILT IN means a mechanism exists. The policy inside it (which tools, which limit, which schema, which role) is still yours to write, and it is the part that matters.

Diagram: The decision for one job · four questions, asked in this order

Read left to right, once for each of the eight jobs. Each column is one question. A job can leave the path at the first or the second column and stay in your code. Only a job that passes both reaches the choice of an owner. This is the method of this page, not a rule from any vendor. Click any box.

  1. 1 · Is it a problem you have?
    1. Name the cause the product removes: Say in one sentence what hurts today. If the sentence starts with "we might", the job stays where it is.
  2. 2 · Does it name your business?
    1. Split the job: mechanism and policy: The mechanism is the same in every company and can be adopted. The policy names your roles, your rules and your limits, and stays with you.
  3. 3 · Which owner fits the conditions?
    1. A library, in your process: Fits when the product exists for your language and you can follow its upgrades. You keep the servers and the data.
    2. A service, at a vendor: Fits when the data may be stored there and an outage of the vendor is acceptable for this job.
    3. Your own code: Fits when the mechanism is small, already written and tested, or must sit next to your business data.
  4. 4 · How will you know it still works?
    1. Run the same golden cases again: The cases and the expected answers are yours and do not depend on the owner. A change of owner is accepted when they pass again.
    2. Write the date and the condition: Record what was decided, on which reading of the documentation, and which condition would reopen the decision.

The table applies the method to each job. The conditions are this site's guidance, built from the job chapters. They are not vendor claims.

JobAdopt whenBuild whenThe question to ask a vendor
1 Context assemblyRuns are long, and compaction and caching are the workThe rule for what may enter the context depends on role or regulationWhat do you add to my prompt, and can I print the final request?
2 Tools, and where they runThe agent needs general tools and a sandbox for themThe tools are business tools, and a narrow set is the safetyIn which process and on which network does each tool run, and what is the default?
3 Control flowThe loop is ordinary, and one limit you set is enough controlThe loop carries rules you must read: a check between steps, a fixed order of stagesWhat stops the loop if the model never stops asking, and what is that default?
4 State, durabilityRuns wait for hours or days, branch, or must survive a deployOne short run in one process, or state that must sit next to your business rowsWhat survives a crash in the middle of a run, whose database holds it, and can an upgraded version read it?
5 Identity, approvalThe pause mechanism, a secret store, a policy engineAlways, for the policy: who may do what, who approves, where it is recordedWhere do I check who the caller is? Does your policy cover tools that run in my application?
6 Output contractChecking the shape against a schemaChecking the meaning, and what happens when the check failsCan the final answer be bound to a schema, and what happens when validation fails?
7 Reliability, costRetry, fallback and routing across modelsThe budget number, and every side effect that must happen onceIs there a limit in money or only in calls? What does a run do at the limit, and in your outage?
8 Observability, evalsStoring traces and the screens to read them; the eval runnerThe golden cases, redaction before a trace leaves, the join with business recordsWhere do traces go by default, what do they contain, and can I export them in an open format?

Read the two middle columns down the page. The "adopt when" column is machinery: stores, loops, pauses, screens. The "build when" column is the list from section 3 again. The method does not produce that split. The map does, and the method only makes a team look at it before signing.

5. The decision for the quote agent#

Apply it to the three proposals. What follows is a design for the running example, argued from its conditions. It is not a recommendation for another team, and each row names the condition that would change it.

The conditions are these. The production service is written in Java. The agent uses three business tools and no shell. The run waits at one gate, for an underwriter. The context holds data about a broker's client. A premium-tier quote needs a senior underwriter. And the team has read its own loop, which is the condition that makes the lower rungs cheap for it.

JobOwner in this designThe reasonWhat would change it
The callAdopt: the provider SDK, rung 2Request plumbing is the same in every company, and the SDK exists for JavaNothing likely
1 ContextKeep. Adopt prompt caching as a feature of the APImay_read decides which document a role may see. That rule is the insurer'sRuns long enough to need compaction
2 ToolsKeep the tools. Adopt MCP as the way to deliver themThree business tools, and one tool server per stage in the full build. No sandbox is needed, because no tool runs a commandA task that needs files or a shell
3 Control flowKeepThe limits are the product: 6 turns for prefill and 10 for underwriting in the full build, with code checks between stagesA loop with no rules of its own
4 StateKeep, in the database the team already operatesOne gate, one wait. The full build writes the state change and the outgoing event in one transaction, which needs both in one databaseMany long waits or branches: then a graph runtime or a durable execution service, in a language the product supports
5 Identity, approvalKeepRoles, the senior-underwriter rule and the approval record are insurance rules. No cell on the map offers themNothing. Only the pause mechanism could move
6 OutputAdopt the schema check from the API. Keep the check of meaningThe shape is generic. The tier floor is notNothing likely
7 Reliability, costAdopt the SDK's retries. Keep the budget and the outbox. The full build also keeps a model gateway of its ownA retry is generic. A budget per quote and "never sent twice" are notMany providers, or many teams sharing one spend limit: then a gateway product
8 Observability, evalsKeep the golden set. Emit traces in a standard format, so the store can change laterThe full build joins traces with business rows in one database. The format keeps the door openTrace screens that cost more to maintain than a product costs to adopt

So the first proposal wins in this room, and for a reason that can be stated: every job the other two would take over is one this agent hardly uses or cannot hand off. The framework's built-in pause would replace three functions the team has already written and tested, at the price of a second vocabulary and a saved-state format. Language narrows it further: in the baseline, one of the seven framework rows lists Java among its languages. The managed platform would remove one service and one database, and its sandbox would sit nearly unused.

Change one condition and the result changes. A Python team with no harness yet, facing many branches and long waits, reads the same map and reaches rung 5. A team whose agent works for hours in a sandbox on data that is not regulated reaches rung 6. The map is the same for all three. The conditions differ.

One artefact appears in every version of this decision, on every rung: the test of whether the agent is right. The command below runs the miniature's golden set. It shows the one check that must pass again after any change of owner. A pass proves that five cases still get the expected decision from the scripted model. It does not prove anything about a live model, or about a sixth case.

/opt/homebrew/bin/python3 evals.py
PASS  naples-complete
PASS  tampa-missing-facts
PASS  unknown-address
PASS  miami-1925
PASS  injected-instruction
score: 5/5 = 1.00

This was run for this page from examples/quote-email/, against the scripted model. If the team ever moves a job to a library or a vendor, these five cases move with it unchanged, and that is how the team will know the move kept the behaviour.

6. A product that is not on the map#

A new product will be announced before this page is a month old, and the map will not have its row. The row can be written in ten minutes, because the columns do not change.

Open the product's documentation and ask one question per job: what does it add to my prompt, where do tools run, what stops the loop, what survives a crash, where is the caller checked, how is the output typed, what is retried and limited, where do traces go. Frameworks has the eight questions in full, and Managed platforms adds seven for a service that holds your state.

Two habits keep the row honest. Write "the docs do not say" when they do not, and resist filling the cell from a talk or a memory. And write the date next to the row. Three cells of this page's own baseline were out of date after two days.

7. Back in the review#

The meeting ends differently from the first one. Nobody won on syntax. The team keeps its harness on the provider SDK, writes down the two conditions that would send it up a rung, and puts the date on the decision.

What changed between the two meetings is not the products. It is what the team sees when it looks at one. An agent is a model call in a loop. Around that loop, eight jobs must be done by someone, and they do not go away when someone else does them. Every SDK, coding tool, framework and platform is the same eight jobs with different owners. So the question was never "build or adopt". It is "who runs each job", asked eight times, with the date written beside the answer.