The design review from The ladder meets again. Then, the team had a thirty-line script and three proposals it could not compare. Now the script is harness.py, about 450 lines, and the proposals are back in a new form. One engineer wants to keep the hand-written harness on the provider SDK. One wants to move it onto an agent framework, "because it has checkpoints and approval built in". One wants a managed platform, "so that we stop running servers for this".
Last time each proposal came with a dozen lines of code, and the room compared syntax. This time nobody shows code. The first question comes from the engineer who wrote the approval gate: "On your proposal, who runs job 5?" The framework's advocate answers at once: the pause is built in, and the check of who the caller is stays in our code. The platform's advocate needs a minute, and then says the same thing in other words.
That exchange is new. The team can now read a proposal as eight answers, one per job, and two proposals that looked different turn out to agree on the job that matters most to an insurer. What the team still lacks is the whole picture: every product it has heard named, on one sheet, read the same way. And then a way to choose that does not end in "which one is best".
You do not choose a product for an agent. You choose an owner for each of eight jobs. A product is a bundle of such choices that someone else made, and the decision is which bundle leaves the right jobs in your hands.
1. Why the products cannot be sorted into boxes#
The obvious way to draw the market is a set of boxes: SDKs here, frameworks there, platforms on top. It fails on the first products you try.
LangChain's documentation calls create_agent an agent harness, and that loop runs on LangGraph, which is a separate product on another rung. The OpenAI Agents SDK and Strands Agents are SDKs by name, and both ship sessions, approval and tracing, which is framework work. Amazon Bedrock AgentCore is a managed platform whose harness module is, according to its documentation, powered by Strands Agents. LangSmith is a tracing product that also deploys agents. A box with a wall around it cannot hold any of these.
So this site does not sort products. It asks each one the question from The eight jobs: who runs this job? A product gets one row and eight answers. Each row also names a primary role, the thing the product is mainly for, only so that the rows can be grouped into tables of a readable size. The role is a label for the table. The cells are the content.
Each cell uses one of five words, with a few words of detail in the vendor's own terms.
| Word in the cell | What it means | What it costs you |
|---|---|---|
| YOU BUILD | The documentation leaves the job to your code | Your engineering time, now and at every change |
| BUILT IN | The product ships the job as a library or a feature. You still configure and operate it | Its vocabulary, its defaults and its upgrades |
| INTEGRATES | The product hands the job to a partner product, which the cell names | A second product to run, and the seam between the two |
| MANAGED | The vendor runs the job as a service | Control over where it runs, how it fails and what it stores |
| NOT ITS JOB | The product does not aim at this job. A fact about its scope, not a fault | Nothing, as long as you know another owner is needed |
There is a sixth entry: "the docs do not say". It means the vendor pages read for this site did not answer. It never means the feature is absent.
The diagram shows the idea on one job, job 4: saving a run so it can continue. Four different owners can hold it. One part stays with the insurer under all four.
Diagram: One job, four possible owners · and the part that stays with you
Read left to right. The job on the left is job 4 from the quote example: save the run, so it can continue after the underwriter answers. The middle column shows the four owners a product map can name. Whichever one you pick, the box on the right is still yours. Click any box.
- 1 · A job that must be done
- Save the run, continue it later: Job 4. The reply waits for a person, perhaps for two days. The run must be saved, the process must be free to exit, and another process must continue it.
- 2 · Who can run it
- You build it: The job is your code and your database. You can read every line, and you maintain every line.
- Built in: a library does it: The product ships the job. You still pass it a database, operate it, and follow its upgrades.
- Integrates: a named partner does it: The product hands the job to a second product. You now run two, and the seam between them.
- Managed: a vendor runs it: The vendor stores the session as a service. You give up control over where the state lives and what happens in an outage.
- 3 · Yours under every owner
- Make a repeated step harmless: Every owner in the middle column can run a step again after a crash or a resume. Only your code knows that the reply email must not go out twice.
Three limits apply to every table below, and they belong before the tables, not after. The cells were read on vendor documentation on 2026-11-16, or come from this site's checked baseline of 2026-11-14 where a row says so. A cell is a reading of documentation, not a test: nothing here was installed and measured for this page. And the set of products is a selection of the ones this series discusses. It is not a census of the market, and the order of rows is not a ranking.
2. The ecosystem map, in five tables#
The ecosystem map is the full table of products by jobs. It is wide and long, so it is split by primary role. Each product page of this site shows only its own rows, in more detail and with the source of every cell. Where a table here shortens a cell, the product page has the full wording.
Provider SDKs and agent SDKs#
These are rungs 2 and 3: a library that wraps the call, and a library that wraps the loop. The first three rows are from SDKs. The next two carry "SDK" in their names and are read in detail on Frameworks. The last row is a model API that can keep state on the server, taken from the baseline.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
| Anthropic client SDKs · provider SDK | YOU BUILD: you send every message on every call | YOU BUILD: the schema and the function, in your process | YOU BUILD: the loop and its stop | YOU BUILD | YOU BUILD | BUILT IN: structured outputs with a JSON Schema | BUILT IN for one request: two retries, a time limit, token counts. A budget across turns: YOU BUILD | BUILT IN: logging; one trace span per request on the Python page. Evals: YOU BUILD |
| The tool runner (beta) · a helper inside the provider SDK | BUILT IN: the history of the run, optional compaction | BUILT IN: derives the schema and calls your function, in your process | BUILT IN: the loop. A turn limit only if you set max_iterations | YOU BUILD | YOU BUILD | The docs do not say | As the provider SDK. A budget: YOU BUILD | The docs do not say |
| Claude Agent SDK · agent SDK | BUILT IN: automatic compaction, skills and memory files | BUILT IN: built-in tools, MCP, in-process custom tools, a sandbox option. Isolation is your deployment | BUILT IN: the loop, max_turns, subagents, hooks | BUILT IN: sessions on local disk, resume and fork, an adapter for external storage | BUILT IN: permission modes, allow and deny rules, a callback, hooks. The end user's identity: the docs do not say | BUILT IN: output_format with a JSON Schema | BUILT IN: max_budget_usd on an estimate, fallback_model. Side effects that must happen once: the docs do not say | BUILT IN: OpenTelemetry export when you switch it on. Evals: the docs do not say |
| OpenAI Agents SDK · agent SDK | BUILT IN: instructions, sessions, an input filter | BUILT IN: function tools, hosted tools, MCP, sandbox agents | BUILT IN: the loop, handoffs | BUILT IN: sessions on several stores, RunState. INTEGRATES: Dapr, Temporal, Restate, DBOS | BUILT IN: guardrails, needs_approval. Authorization: YOU BUILD, inside the tool | BUILT IN: output_type | BUILT IN: max_turns, usage tracking, opt-in retries. Fallback: the pages read do not say | BUILT IN: tracing, on by default. Evals: not in the pages read |
| Strands Agents · agent SDK | BUILT IN: conversation managers | BUILT IN: tools, MCP, Docker and SSH sandboxes | BUILT IN: the loop, optional limits | BUILT IN: session managers for files and S3. Durable execution: the pages read do not say | BUILT IN: HumanInTheLoop, CedarAuthorization | BUILT IN: structured_output_model | BUILT IN: retries on throttling, limits you pass | BUILT IN: OpenTelemetry traces |
| OpenAI Responses and Conversations API · model API with optional stored state | MANAGED, optional: server-side compaction | MANAGED: built-in tools, remote MCP | The docs do not say who runs the loop | MANAGED, optional: responses stored 30 days by default; conversation objects persist | NOT ITS JOB for your own tools. Approval settings exist for remote MCP tools | BUILT IN: structured outputs | BUILT IN: prompt caching | Not confirmed on a page read |
The first two rows say YOU BUILD in columns 4 and 5, and so does every provider SDK. A library that wraps a request has no place to keep a run and no way to know your users. The step from the second row to the third is the largest on the map for a single install: the Agent SDK fills six columns that the tool runner leaves empty. Its price is on SDKs: a long-lived process per session, in Python or TypeScript only.
Coding harnesses#
A coding harness is a finished harness for software work, so every job is already built. The cells below are feature names from the translation table on Coding harnesses, read here as BUILT IN. That page has the files, the flags and the defaults.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
| Claude Code · coding harness | BUILT IN: CLAUDE.md, automatic compaction, skills | BUILT IN: tools, MCP. A sandbox for shell commands, off by default | BUILT IN: plan mode, subagents, claude -p; --max-turns in scripted runs | BUILT IN: --continue, --resume, /rewind | BUILT IN: six permission modes; allow, ask and deny rules; hooks | BUILT IN: --json-schema | BUILT IN: /usage, --max-budget-usd | BUILT IN: JSONL transcripts, OpenTelemetry export |
| Codex CLI · coding harness | BUILT IN: AGENTS.md, skills. Compaction: two hook events; beyond that the docs do not say | BUILT IN: tools, MCP, sandbox_mode with three levels | BUILT IN: subagents, codex exec. Plan mode and turn limits: the docs do not say | BUILT IN: codex resume | BUILT IN: approval_policy, hooks | BUILT IN: --output-schema | Token counts in the turn.completed event | BUILT IN: an event stream with --json. Tracing: the docs do not say |
| GitHub Copilot CLI · coding harness | BUILT IN: instruction files, automatic compaction, skills | BUILT IN: tools, MCP, /sandbox enable, trusted directories | BUILT IN: plan mode, autopilot mode, subagents, copilot -p | BUILT IN: --continue, --resume | BUILT IN: --allow-tool, --deny-tool, hooks | The docs do not say | BUILT IN: /usage, /context | BUILT IN: session state and logs on disk. Tracing: the docs do not say |
| Antigravity CLI · coding harness | BUILT IN: rules files, automatic compaction, skills | BUILT IN: tools, MCP, --sandbox, off by default | BUILT IN: --mode plan, subagents, agy -p. Turn limit: the docs do not say | BUILT IN: --continue, /resume, /rewind | BUILT IN: three modes; allow, ask and deny rules; hooks | BUILT IN: --json-schema | BUILT IN: /usage, /credits | BUILT IN: transcript.jsonl. Tracing: the docs do not say |
| Cursor and Cursor CLI · coding harness | BUILT IN: rules files, skills. Compaction: one hook event; beyond that the docs do not say | BUILT IN: tools, MCP, --sandbox enabled or disabled | BUILT IN: plan mode, subagents, agent -p. Turn limit: the docs do not say | BUILT IN: agent resume, agent ls | BUILT IN: permissions.allow and deny, hooks | The docs do not say | The docs do not say | BUILT IN: a transcript file. Tracing: not read |
| Grok Build · coding harness | BUILT IN: AGENTS.md, automatic compaction, skills | BUILT IN: tools, MCP, --sandbox with a profile, off by default | BUILT IN: /plan, subagents, grok -p. Turn limit: the docs do not say | BUILT IN: --continue, --resume | BUILT IN: three modes, [permission] rules, hooks | The docs do not say | BUILT IN: /usage | BUILT IN: /transcript, /export. Tracing: the docs do not say |
Columns 1 to 5 are full in every row, and columns 6 to 8 are where "the docs do not say" collects. That is the order in which a developer at a keyboard needs the jobs. A production agent needs them in the reverse order. Column 5 is also the one to read twice: the gate in every row asks the developer who is present, and that is a different person from the underwriter who approves a quote.
Agent frameworks and graph runtimes#
These are rungs 4 and 5: the loop as a library, and a runtime that saves the state as the run proceeds. The rows are from Frameworks, shortened. That page also lists the defaults each product sets for you.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
LangChain create_agent · agent framework | BUILT IN: system_prompt, middleware. Summarization only if you set a trigger | BUILT IN: tools from functions. The shell tool runs on the host unless you pick DockerExecutionPolicy | BUILT IN: the loop, on LangGraph | BUILT IN once you pass a checkpointer | BUILT IN: HumanInTheLoopMiddleware. Caller identity: the pages read do not say | BUILT IN: response_format | BUILT IN: retry, fallback and call-limit middleware | INTEGRATES: LangSmith |
| LangGraph · graph runtime | BUILT IN: state, store. Prompts are yours | BUILT IN: a prebuilt ToolNode class. The quickstart writes the tool node by hand. Sandbox: the pages read do not say | BUILT IN: the graph | BUILT IN: checkpointers for memory, SQLite, PostgreSQL, MongoDB, Cosmos DB | BUILT IN: interrupt. Identity: the pages read do not say | The pages read do not say | BUILT IN: RetryPolicy, node timeouts. Token budget: the pages read do not say | INTEGRATES: LangSmith |
| LlamaIndex · agent framework, data-centric | BUILT IN: retrieval over your data, memory classes | BUILT IN: tools from functions. Sandbox: not re-read today | BUILT IN: event-driven Workflows | YOU BUILD the saving, with your own store. INTEGRATES: DBOS, to survive a crash | BUILT IN: events that wait for a person. Identity: the pages read do not say | BUILT IN: output_cls | BUILT IN: retry_policy on steps. Agent turn limit: the pages read do not say | BUILT IN: instrumentation, evaluation modules. INTEGRATES: Arize Phoenix, Langfuse, Opik |
| CrewAI · agent framework | BUILT IN: role templates, memory. Summarizes at the context limit by default | BUILT IN: tools, MCP. Code execution is deprecated | BUILT IN: Crews and Flows | BUILT IN: checkpoints to JSON files or SQLite; @persist on Flows. Resuming is a call you make | BUILT IN: human_input, @human_feedback. Identity: YOU BUILD | BUILT IN: output_pydantic, output_json | BUILT IN: max_iter, max_retry_limit, max_rpm. Fallback: the pages read do not say | MANAGED: CrewAI AMP tracing. INTEGRATES: Langfuse, Arize Phoenix and others |
| Pydantic AI · agent framework, typed | BUILT IN: instructions, typed dependencies | BUILT IN: tools, MCP. Sandbox: not confirmed today | BUILT IN: the loop, pydantic-graph | YOU BUILD: you store the message history. INTEGRATES: nine durable execution engines | BUILT IN: requires_approval. Identity: YOU BUILD | BUILT IN: output_type, with a validation retry | BUILT IN: UsageLimits, ModelFallback, retries | BUILT IN: OpenTelemetry, Pydantic Evals. INTEGRATES: Logfire |
| Microsoft Agent Framework · agent framework | BUILT IN: context providers. Compaction is experimental in Python | BUILT IN: tools, MCP. Shell tools: a beta package | BUILT IN: agents and graph workflows | BUILT IN: AgentSession, which you serialize; workflow checkpoints to memory, file or Cosmos DB | BUILT IN: approval_mode. Identity: YOU BUILD | BUILT IN: response_format | YOU BUILD: retry and fallback as middleware. Tool-call limits default to unlimited | BUILT IN: OpenTelemetry. Evals: the pages read do not say |
| Google ADK · agent framework | BUILT IN: instruction templates, compaction. Context caching is experimental | BUILT IN: tools, MCP, OpenAPI | BUILT IN: graph workflows | BUILT IN: session services for memory, a database or Agent Platform; resumable workflows | BUILT IN: require_confirmation, tool authentication | BUILT IN: output_schema | BUILT IN: max_llm_calls, retry on graph nodes. Python fallback: the pages read do not say | BUILT IN: logging, OpenTelemetry traces, adk eval |
Column 3 says BUILT IN in every row, which is what a framework is. Column 5 splits inside its own name: the pause for a person is built in everywhere, and the check of who the caller is comes back to your code in almost every row. Column 4 hides two promises under one heading. "The next message remembers the last one" is a session store. "The run survives a crash in the middle" is durable execution, and three rows answer that one with YOU BUILD or a partner.
Managed platforms#
This is rung 6: the vendor runs the loop, the sandbox and the stored session. The rows are from Managed platforms. They describe one mode of two products: the harness module of AgentCore, and prompt agents on Foundry.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
| Claude Managed Agents (beta) · managed harness | MANAGED: prompt caching, compaction | MANAGED: a container per session, or a worker you host. Custom tools: YOU BUILD | MANAGED | MANAGED: session history, sandbox checkpoint | MANAGED: permission policies, vaults | The docs do not say | MANAGED: session budget, automatic retry of transient errors | MANAGED: session viewer, usage, OpenTelemetry tracing |
| OpenAI Agents API (beta) · managed harness | MANAGED: context compaction | MANAGED: a hosted sandbox, or an executor you host. Function tools: YOU BUILD | MANAGED | MANAGED: session state | MANAGED: vaults, automatic review | The docs do not say | MANAGED: sessions resume. Retrying a failed request: YOU BUILD | MANAGED: tracing on by default, export |
| Amazon Bedrock AgentCore, harness · managed harness | MANAGED: truncation by sliding window or summarization | MANAGED: a microVM per session, Gateway, Code Interpreter, Browser. Inline functions: YOU BUILD | MANAGED: the harness. On Runtime alone, the loop is your code | MANAGED: Memory, short and long term | MANAGED: Identity, Policy | The docs do not say | MANAGED: limits on iterations, time and tokens | MANAGED: Observability, Evaluations |
| Foundry Agent Service, prompt agents · managed harness | MANAGED: input truncated to the model's context size | MANAGED: Code Interpreter sandbox, Toolboxes, MCP. Function calling: YOU BUILD | MANAGED. A hosted agent runs your code instead | MANAGED: Microsoft's storage, or your own Azure resources | MANAGED: Entra agent identity, role-based access, content filters, approval for MCP tools | The docs do not say | YOU BUILD: retry rate-limit errors with backoff | MANAGED: tracing, metrics, evaluations |
| Google Agent Runtime · managed hosting for your agent code | The docs do not say | MANAGED: runs the container you deploy. Code Execution sandbox | The docs do not say | MANAGED: Sessions, Memory Bank | MANAGED: Agent Identity, Agent Gateway | The docs do not say | The docs do not say | MANAGED: tracing, evaluation |
Column 2 has the same shape in four rows: a managed sandbox, then YOU BUILD for your own tools. A vendor can run a shell. It cannot run the insurer's underwriting rules. Column 6 is empty in all five rows: none of the agent definitions read has a field for an output schema, so the check of the final answer stays in your application. And the word "managed" covers two offers: a vendor that runs its harness for you, and a vendor that hosts a loop you wrote with a framework. The last row is the second kind.
Products that take one job#
Some products do not run the loop at all. They sit beside it and take one job, or one and a half. A row of NOT ITS JOB is not a weak product. It is a narrow one, and it combines with any rung. The Temporal row is from Frameworks and Managed platforms. The other four rows come from the baseline, and each product's front page was read again on 2026-11-16.
| Product and primary role | 1 Context | 2 Tools | 3 Control flow | 4 State, durability | 5 Identity, approval | 6 Output | 7 Reliability, cost | 8 Observability, evals |
|---|---|---|---|---|---|---|---|---|
| Temporal · durable execution platform | NOT ITS JOB | NOT ITS JOB: a tool call is one of your Activities | BUILT IN: Workflows as code | BUILT IN: Event History and replay. Temporal Cloud: MANAGED storage; your Workers stay with you | NOT ITS JOB. A Signal lets a Workflow wait for a person | NOT ITS JOB | BUILT IN: Activity retries, unlimited attempts by default | BUILT IN: metrics, tracing, logging |
| LangSmith, with Agent Server · tracing and eval platform that also deploys | NOT ITS JOB at run time. Prompt engineering is a named product area | NOT ITS JOB | NOT ITS JOB: the graph is your code | MANAGED: Agent Server, with persistence and a task queue. Cloud or hybrid | Agent Server: custom authentication and authorization handlers, per the baseline | NOT ITS JOB | An LLM Gateway is named. Details: the docs do not say | BUILT IN: tracing, dashboards, alerts, offline and online evaluation |
| Langfuse · tracing and eval platform, open source | BUILT IN in part: prompt management | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | BUILT IN: an AI Gateway that tracks spend per user, key and model | BUILT IN: tracing, evaluation, datasets |
| Arize Phoenix · tracing and eval tool, open source | BUILT IN in part: prompt management, a playground | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | NOT ITS JOB | BUILT IN: tracing, evaluation, datasets and experiments |
| LiteLLM · model gateway | NOT ITS JOB | NOT ITS JOB. Its README also names an MCP gateway | NOT ITS JOB | NOT ITS JOB | BUILT IN: virtual keys, guardrails | NOT ITS JOB | BUILT IN: retries, fallbacks, cooldowns, timeouts, load balancing, spend tracking | INTEGRATES: logging to other tools |
Three jobs can be bought alone: job 4 from a durable execution service, job 7 from a gateway, job 8 from a tracing product. No row on this table takes job 2 or job 6. The two filled cells in column 5 are about keys for model calls and access to a deployment server. Neither is about who may approve a quote. The NOT ITS JOB cells in the four baseline rows are this site's reading of each product's scope. No vendor page states them.
Durable execution has more names than one row. The cells above name DBOS, Restate, Dapr and others as partners of frameworks. DBOS describes itself as "a library for building reliable programs", and Restate as "a lightweight runtime to turn AI agents, workflows, and backend services into durable processes". Neither was read beyond its front page, so neither has a row.
Protocols: the connectors between the rows#
Three names come up next to these products and are not products. Nobody runs a job by adopting a protocol. A protocol decides how two owners of jobs talk to each other, which matters most on the day one of them is replaced.
| Protocol | What it connects | The job it touches | Status as read |
|---|---|---|---|
| MCP, the Model Context Protocol | A harness to tool servers | Job 2. Home: MCP | Specification version 2026-07-28 |
| A2A, the Agent2Agent protocol | One agent system to another that it cannot see inside | Jobs 2 and 3, across a company boundary. See Many agents | Version 1.0, under the Linux Foundation |
| OpenTelemetry, GenAI semantic conventions | A harness to any trace store | Job 8. See Observability and evals | Marked "Development", not stable |
How the map was built, and which names changedThree sources, one rule for conflicts, five renames
Each cell has one of three sources. Rows that a product page of this site covers were copied from that page, which read the vendor's documentation on 2026-11-16. Rows that no product page covers come from the baseline of 2026-11-14, and the vendor's front page or the page behind the cell was read again on 2026-11-16. A cell that the baseline marks as unknown is written as "the docs do not say".
Where a product page and the baseline differ, the product page wins, because it is the later reading. Three such cells are on the map. LlamaIndex, job 4: the baseline said built in, and the page read on 2026-11-16 says "There is no built-in checkpointer to enable." Foundry Agent Service, job 6: the baseline said managed, and the schema setting was found only in a workflow designer in preview. Pydantic AI, job 4: the baseline named three partner engines, and the overview now names nine. Two days were enough to move three cells, which is the best argument for reading a cell as dated.
One baseline cell is left out: a line about turn limits and token budgets for Strands Agents was not confirmed on the vendor's page, so the row shows what the product page read.
Names in this field change, and an old name in a proposal is a reason to check the date of the proposal. From the baseline: AutoGen's README says it is archived and no longer maintained, and names Microsoft Agent Framework as the successor. Semantic Kernel's README says it "is now Microsoft Agent Framework". The OpenAI Assistants API was shut down on 2026-08-26, and the Responses and Conversations APIs replace it. Vertex AI Agent Engine became Agent Runtime on 2026-04-22. Antigravity CLI replaced Gemini CLI for free-tier and Google AI Pro and Ultra users on 2026-06-18.
The fact log for this page lists every cell that does not come from a product page, with its address and the date read.
3. The same map, read by column#
Read by row, the map describes products. Read by column, it describes the jobs, and that reading is the one a decision needs. For each job: who usually runs it, and what stays with you in every row.
Job 1, context assembly. Almost every product that runs a loop also builds the request: BUILT IN from rung 3 up, MANAGED on rung 6. What they build is the mechanism: history, compaction, caching. Which data may enter the context is not in any cell. In the miniature that decision is assemble_context and may_read, and it stays where the knowledge of the insurer's data is.
Job 2, tools and where they run. The map splits this job in two. General tools and a sandbox for them are widely BUILT IN or MANAGED. Your own tools are YOU BUILD in every row that mentions them, on every rung, because their logic and data are your business. The model asks, the harness decides and calls, the tool does: a product can supply the middle actor, never the last.
Job 3, control flow. This is the most adopted job on the map. Every agent SDK, framework and managed harness runs the loop. What stays is the number: the turn limit that fits your task. Frameworks lists defaults from 10 to 10,007 and several of "no limit". The loop is theirs, and the stop is still your decision.
Job 4, state, memory and durability. Every answer appears in this column: YOU BUILD, BUILT IN with your database, INTEGRATES with a named engine, MANAGED. It is the column where products differ most, so it is the one to read first when a run must wait for a person. One thing stays in every row: making a repeated step harmless. Each vendor that resumes work says so in its own words, and in the miniature that is send_once.
Job 5, identity, permissions and approval. The pause for a person is BUILT IN or MANAGED almost everywhere. The rest of the job is not. Who the caller is, which role may run which tool, who may approve and where that is recorded: the map shows YOU BUILD or silence for these in most framework rows, and policy mechanisms without the policy in the managed rows.
Job 6, the output contract. Checking the shape of the answer is BUILT IN in most rows from the provider SDK upward, and the documentation of all five managed platforms is silent on it. Checking the meaning is in no cell at all. A quote in the standard band for a house in a flood zone passes every schema.
Job 7, reliability and cost. Retrying one request is BUILT IN at rung 2. Limits on calls are common above it. A limit in money is rare: a few cells name a budget, and one of those says it is an estimate. The bill is yours in every row, and so is the record that a side effect happened once.
Job 8, observability and evals. Traces are the easiest job to hand over: nearly every row offers one, and three products on the single-job table are built around it. Evals are offered as machinery: a runner, a scorer, a dataset store. The cases and the definition of "correct" are in no product, because they are knowledge about your business.
Put the eight "what stays" sentences side by side and one list appears. It is the same list Managed platforms found at the top of the ladder: the logic of your tools, the rule about what the agent may do, the person who approves, the judgment of results, and the bill. No rung removes any of the five.
4. How to decide, job by job#
The map does not say what to choose. It says what each choice hands to whom. The choice needs conditions, and the conditions come from the team's own situation, so start there.
Take the framework proposal from the opening. Its advocate named a real problem: the reply must wait for an underwriter, perhaps for two days, and the team wrote checkpoint, load_checkpoint and send_once to make that work. Is that a reason to adopt? It depends on three things the proposal did not state. How many such waits will there be: one gate, or many branches with long pauses? What language is the service written in, and does the product exist for it? And how much of the saved state would then be in the product's format? A run that is paused today must be resumable after next month's upgrade.
Now the managed proposal. Its advocate also named a real problem: someone operates the service, the database and the sandbox. The questions here are different. May the broker's email and every tool result be stored at a vendor? What does a session do while the vendor is unavailable? And how much of the service would be left anyway, since all three business tools run in the insurer's application on every platform?
Neither set of questions has a general answer, which is why this page has no ranking. Each direction has a cost, and the cost is of a different kind.
| Direction | What you gain | What you pay | When the cost is highest |
|---|---|---|---|
| Build the job | Code you can read, test and change; the request is a variable you can print | Writing it, and then maintaining it at every model and API change | When the job is the same in every company, so your version has no advantage |
| Adopt a library (BUILT IN, INTEGRATES) | The repeated half of a harness, written and tested by others | A second vocabulary, defaults you did not choose, upgrades on someone else's schedule, saved state in its format | When your code touches much of its surface, or runs live long |
| Adopt a service (MANAGED) | No process, sandbox or session store to operate | Less control over where state lives, what the network allows and what happens in an outage; a dependency for every run | When the data is regulated, or the agent mostly uses your own tools |
Three rules turn those costs into a method.
The first is the rule from The ladder, made exact: start at the lowest rung that removes a real problem you have. "We keep rewriting the retry code" is a problem, and rung 2 removes it. "A person must approve, and may take days" is a problem, and a graph runtime or a durable execution service removes it. "We might need it later" is not a problem yet, and climbing for it buys the cost without the gain.
The second: decide per job, not per system. Nothing on the map forces one owner for all eight. A provider SDK for the call, a durable execution service for the long wait, a tracing product for job 8 and your own code for identity is an ordinary combination. The single-job table exists because of it.
The third: a job that names your business stays with you, whatever the cell says. BUILT IN means a mechanism exists. The policy inside it (which tools, which limit, which schema, which role) is still yours to write, and it is the part that matters.
Diagram: The decision for one job · four questions, asked in this order
Read left to right, once for each of the eight jobs. Each column is one question. A job can leave the path at the first or the second column and stay in your code. Only a job that passes both reaches the choice of an owner. This is the method of this page, not a rule from any vendor. Click any box.
- 1 · Is it a problem you have?
- Name the cause the product removes: Say in one sentence what hurts today. If the sentence starts with "we might", the job stays where it is.
- 2 · Does it name your business?
- Split the job: mechanism and policy: The mechanism is the same in every company and can be adopted. The policy names your roles, your rules and your limits, and stays with you.
- 3 · Which owner fits the conditions?
- A library, in your process: Fits when the product exists for your language and you can follow its upgrades. You keep the servers and the data.
- A service, at a vendor: Fits when the data may be stored there and an outage of the vendor is acceptable for this job.
- Your own code: Fits when the mechanism is small, already written and tested, or must sit next to your business data.
- 4 · How will you know it still works?
- Run the same golden cases again: The cases and the expected answers are yours and do not depend on the owner. A change of owner is accepted when they pass again.
- Write the date and the condition: Record what was decided, on which reading of the documentation, and which condition would reopen the decision.
The table applies the method to each job. The conditions are this site's guidance, built from the job chapters. They are not vendor claims.
| Job | Adopt when | Build when | The question to ask a vendor |
|---|---|---|---|
| 1 Context assembly | Runs are long, and compaction and caching are the work | The rule for what may enter the context depends on role or regulation | What do you add to my prompt, and can I print the final request? |
| 2 Tools, and where they run | The agent needs general tools and a sandbox for them | The tools are business tools, and a narrow set is the safety | In which process and on which network does each tool run, and what is the default? |
| 3 Control flow | The loop is ordinary, and one limit you set is enough control | The loop carries rules you must read: a check between steps, a fixed order of stages | What stops the loop if the model never stops asking, and what is that default? |
| 4 State, durability | Runs wait for hours or days, branch, or must survive a deploy | One short run in one process, or state that must sit next to your business rows | What survives a crash in the middle of a run, whose database holds it, and can an upgraded version read it? |
| 5 Identity, approval | The pause mechanism, a secret store, a policy engine | Always, for the policy: who may do what, who approves, where it is recorded | Where do I check who the caller is? Does your policy cover tools that run in my application? |
| 6 Output contract | Checking the shape against a schema | Checking the meaning, and what happens when the check fails | Can the final answer be bound to a schema, and what happens when validation fails? |
| 7 Reliability, cost | Retry, fallback and routing across models | The budget number, and every side effect that must happen once | Is there a limit in money or only in calls? What does a run do at the limit, and in your outage? |
| 8 Observability, evals | Storing traces and the screens to read them; the eval runner | The golden cases, redaction before a trace leaves, the join with business records | Where do traces go by default, what do they contain, and can I export them in an open format? |
Read the two middle columns down the page. The "adopt when" column is machinery: stores, loops, pauses, screens. The "build when" column is the list from section 3 again. The method does not produce that split. The map does, and the method only makes a team look at it before signing.
5. The decision for the quote agent#
Apply it to the three proposals. What follows is a design for the running example, argued from its conditions. It is not a recommendation for another team, and each row names the condition that would change it.
The conditions are these. The production service is written in Java. The agent uses three business tools and no shell. The run waits at one gate, for an underwriter. The context holds data about a broker's client. A premium-tier quote needs a senior underwriter. And the team has read its own loop, which is the condition that makes the lower rungs cheap for it.
| Job | Owner in this design | The reason | What would change it |
|---|---|---|---|
| The call | Adopt: the provider SDK, rung 2 | Request plumbing is the same in every company, and the SDK exists for Java | Nothing likely |
| 1 Context | Keep. Adopt prompt caching as a feature of the API | may_read decides which document a role may see. That rule is the insurer's | Runs long enough to need compaction |
| 2 Tools | Keep the tools. Adopt MCP as the way to deliver them | Three business tools, and one tool server per stage in the full build. No sandbox is needed, because no tool runs a command | A task that needs files or a shell |
| 3 Control flow | Keep | The limits are the product: 6 turns for prefill and 10 for underwriting in the full build, with code checks between stages | A loop with no rules of its own |
| 4 State | Keep, in the database the team already operates | One gate, one wait. The full build writes the state change and the outgoing event in one transaction, which needs both in one database | Many long waits or branches: then a graph runtime or a durable execution service, in a language the product supports |
| 5 Identity, approval | Keep | Roles, the senior-underwriter rule and the approval record are insurance rules. No cell on the map offers them | Nothing. Only the pause mechanism could move |
| 6 Output | Adopt the schema check from the API. Keep the check of meaning | The shape is generic. The tier floor is not | Nothing likely |
| 7 Reliability, cost | Adopt the SDK's retries. Keep the budget and the outbox. The full build also keeps a model gateway of its own | A retry is generic. A budget per quote and "never sent twice" are not | Many providers, or many teams sharing one spend limit: then a gateway product |
| 8 Observability, evals | Keep the golden set. Emit traces in a standard format, so the store can change later | The full build joins traces with business rows in one database. The format keeps the door open | Trace screens that cost more to maintain than a product costs to adopt |
So the first proposal wins in this room, and for a reason that can be stated: every job the other two would take over is one this agent hardly uses or cannot hand off. The framework's built-in pause would replace three functions the team has already written and tested, at the price of a second vocabulary and a saved-state format. Language narrows it further: in the baseline, one of the seven framework rows lists Java among its languages. The managed platform would remove one service and one database, and its sandbox would sit nearly unused.
Change one condition and the result changes. A Python team with no harness yet, facing many branches and long waits, reads the same map and reaches rung 5. A team whose agent works for hours in a sandbox on data that is not regulated reaches rung 6. The map is the same for all three. The conditions differ.
One artefact appears in every version of this decision, on every rung: the test of whether the agent is right. The command below runs the miniature's golden set. It shows the one check that must pass again after any change of owner. A pass proves that five cases still get the expected decision from the scripted model. It does not prove anything about a live model, or about a sixth case.
/opt/homebrew/bin/python3 evals.py
PASS naples-complete
PASS tampa-missing-facts
PASS unknown-address
PASS miami-1925
PASS injected-instruction
score: 5/5 = 1.00
This was run for this page from examples/quote-email/, against the scripted model. If the team ever moves a job to a library or a vendor, these five cases move with it unchanged, and that is how the team will know the move kept the behaviour.
6. A product that is not on the map#
A new product will be announced before this page is a month old, and the map will not have its row. The row can be written in ten minutes, because the columns do not change.
Open the product's documentation and ask one question per job: what does it add to my prompt, where do tools run, what stops the loop, what survives a crash, where is the caller checked, how is the output typed, what is retried and limited, where do traces go. Frameworks has the eight questions in full, and Managed platforms adds seven for a service that holds your state.
Two habits keep the row honest. Write "the docs do not say" when they do not, and resist filling the cell from a talk or a memory. And write the date next to the row. Three cells of this page's own baseline were out of date after two days.
7. Back in the review#
The meeting ends differently from the first one. Nobody won on syntax. The team keeps its harness on the provider SDK, writes down the two conditions that would send it up a rung, and puts the date on the decision.
What changed between the two meetings is not the products. It is what the team sees when it looks at one. An agent is a model call in a loop. Around that loop, eight jobs must be done by someone, and they do not go away when someone else does them. Every SDK, coding tool, framework and platform is the same eight jobs with different owners. So the question was never "build or adopt". It is "who runs each job", asked eight times, with the date written beside the answer.