How AI works · Agents
Workflows or Agents?
Any AI feature with more than one step has to say who chooses the next step, the code or the model. The answer depends on how much the path varies, how cheap the result is to check, what a wrong step costs and how often the task repeats, and most systems that work end up using both.
"Agents are the future" is a slogan, and like most slogans it answers a question nobody asked. The question that matters when you build anything with AI is smaller and far more useful: when a task takes more than one step, who picks the next step, the code or the model?
In a workflow, code picks. The program calls the model for step one, hands the result to step two, and so on. The model fills in each step but never decides what comes next. In an agent, the model picks. It looks at what has happened so far, chooses a tool, reads the result and chooses again, until it judges the task done.
Almost everything else follows from that one assignment of control: why workflows are cheap, testable and easy to audit, why agents cope with situations nobody planned for, and why the systems that hold up in production usually contain both.
So here is the argument. Decide who holds control stage by stage, not product by product, using four properties of the task: how much the path varies, how cheap the result is to check, what a wrong step costs, and how often the task repeats. An agent beats a fixed sequence only where the world pushes back hard enough for it to catch its own mistakes, or where many real requests fall outside the paths you designed. Everywhere else, code should hold the wheel.
One email, two designs
A software company's support inbox receives: "You charged me twice for March and I need it fixed before Friday."
As a workflow, code runs a fixed sequence. A model call classifies the message as billing, shipping, how-to or other. Code routes on the label and looks up the customer's invoices. A model call drafts a reply from a template and the records. Code checks the draft against rules, such as whether the refund is within the limit and the invoice number matches the account, and then sends it or hands it to a person.
As an agent, the model gets the message, a goal and a set of tools: find a customer, list invoices, read a policy, draft a reply, request a refund, hand off to a person. It decides what to look up, whether to look twice and when it's done. Code still sets the limits: which tools exist, a cap on turns, a budget, approval before money moves (How a Coding Agent Actually Works walks through that loop).
On the ordinary double charge, both succeed, and the workflow does it with fewer tokens and a log that reads like its flowchart. The difference shows up on requests nobody drew a box for.
Suppose the customer also says the VAT number on the invoice is wrong. The workflow assigns one label, billing, and its template covers duplicate charges. The VAT complaint vanishes, and the reply looks complete. The agent reads both complaints and has the tools to investigate both. Now suppose the message ends by mentioning a lawyer. The workflow has no legal category and sends a cheerful refund notice. The agent might hand off to a person, or might not, depending on its instructions and that particular run.
That is the whole contrast in miniature. A workflow fails silently, along the path it was given. An agent fails by variance: it can take a different route on the same input, and occasionally a wrong one.
The arithmetic that decides
Here is the part most debates skip.
Guess first
A chain has ten steps, and each step is right 97 percent of the time. How often is the whole chain right?
About 74 percent. Five steps give 86 percent; twenty give 54. A step that's right 97 times in 100 sounds dependable, and the product still falls quickly. Real chains do worse, because errors aren't independent: a wrong classification at step one spoils everything after it.
An agent has the same problem, usually with more steps. Give our support agent eight turns at the same 97 percent and, if it never fixes a mistake, it's right only 78 percent of the time, worse than the five-step workflow.
So an agent wins only if it notices and repairs some of its own errors. In this example, it has to recover about 37 percent of them just to match the workflow. Recover 40 percent and it's right about 86 percent of the time; recover 60 percent and it's at 91.
What decides that recovery share is mostly whether the world pushes back. A "no such invoice" error, a refund rejected as over the limit, a failed format check, a failing test: each hands the agent a concrete reason to change course. Errors that look like plausible answers, such as a misread intent, give no signal at all, and research on models reviewing their own reasoning without outside information finds they rarely catch those (Same Model, Different Results).
This is why coding agents work so well: compilers and test suites push back constantly. And it predicts where agents help least: wherever nothing outside the model can say the answer is wrong. That prediction is an inference from the mechanism, not a measured result.
An agent is only as good as the feedback it gets from the world. Where nothing pushes back, it's a longer chain with more ways to drift.
The token bill is not the cost
The usual objection to agents is price, and on tokens alone it looks serious. Every turn of an agent re-reads everything before it. With illustrative but realistic numbers, the support workflow costs about 3 cents per request in model fees, and the agent about 12 cents, nearly four times as much. Prompt caching, where the provider charges much less for text it has already processed, brings the agent down to about 5 cents, 1.6 times the workflow.
Now add the people. Say a case escalated to a human costs $4, and a wrong reply that reaches a customer costs $25 in rework, refunds and goodwill. On requests that fit the designed paths, the expected cost is about $2.08 per request for the workflow and $2.16 for an agent that recovers 40 percent of its errors. The model bill is under 2 percent of the first and under 6 percent of the second.
In other words, what decides the cost is how often each design is right and what happens when it's wrong. The token bill only takes over at volume on cheap tasks: at a million requests a month, $30,000 of model spend against $118,000 uncached starts to matter.
The real swing factor is how many requests leave the designed paths. In this example, with the agent recovering 40 percent of its errors, it becomes the cheaper design once just a few percent of requests are off-path, mostly because the workflow forces many of them down the wrong branch. That number is fragile: the example was built near a tie, and a workflow with its own retry step, or an agent that recovers less, moves the crossover a long way. Which is exactly the point. The decision turns on three quantities you can measure: how often real inputs leave the designed paths, how much of its own error each design recovers, and what a wrong reply costs. Nobody else's benchmark can tell you those.
Testing, audit and the problem of the second run
Variance has a cost the arithmetic above doesn't capture. A workflow takes the same path every time. An agent can take different paths on the same input. A 2024 benchmark of agents serving simulated customers, τ-bench, introduced a measure called pass^k, the chance of succeeding on all of k repeated attempts at the same task. Even the best agents of that year solved under half its tasks, and under a quarter on all eight retail trials. Those were older models, but the lesson stands: an average success rate hides how often the same customer, writing twice, gets treated differently.
That changes how you test. A workflow has seams. Each model step takes defined input and returns defined output, so you test the classifier against labeled examples and the drafting step against a rubric, and a failure points at a step. An agent's steps aren't fixed, so you grade the outcome (does the refund exist in the database?) rather than the path, and you run each case several times. A 75 percent success rate per run means only a 42 percent chance of passing three runs in a row (Build an Evaluation From Your Own Work).
Audit splits the same way. A workflow's path is known before it runs, so a reviewer can approve the flowchart. An agent's path is known only afterward, so you approve its bounds in advance and keep a trace of every tool call. In regulated work, "we approved the path" is an easier sentence to defend than "we approved the limits, and here is the log".
What better models change, and what they don't
Better models push toward agents. They're more reliable per step, which weakens the compounding penalty, and they can carry longer tasks. METR, a research group that measures this, reports that the length of software task a model completes half the time, measured in human-expert hours, has doubled about every seven months since 2019. The length it completes 80 percent of the time is four to six times shorter, and that's the one a product team should read, because a feature that works half the time doesn't ship.
The practical effect is that the steps a designer hard-codes shrink. Five model calls become two. The router disappears when the model picks its own tools. That hand-built structure is scaffolding, and the right habit is to try removing it with every new model and keep the removal if your evaluation set, your own collection of test cases with known good answers, still passes.
Several things don't move with the model at all:
- Permissions. What an agent may touch is a fact about your organization, and a more capable agent uses whatever access it has more effectively.
- Irreversible effects. A better model lowers the error rate. It doesn't unsend a message (Reversible Work and Irreversible Effects).
- Audit. Someone still has to say what happened.
- Repetition. A stable path that runs thousands of times a day costs nothing per decision as code, and returns the same answer every time.
- Cheap checks. A total that must add up beats asking a model whether the output looks right, at any capability level.
My view: agents for exploring, code for what must not vary
The strongest case for "agents are the future" deserves a fair hearing. Labs train models inside agent loops, so models are increasingly good at exactly the part a hand-built chain gives to code. A workflow with fifty branches and a router retrained every quarter is a burden an agent with good tools avoids. Where the path varies and results are cheap to check, as in coding, the default has already moved to agents.
All true. And the share of tasks that justify a long hand-built chain is probably shrinking. But "models will do more of the deciding" and "what must not vary belongs in code" are statements about different layers. Even the most autonomous agent runs inside a loop, permissions and budgets written in ordinary code. The builders say so themselves: Anthropic advises the simplest design that works, and OpenAI advises adding autonomy in steps.
So most of what works is a hybrid, in one of two shapes.
A workflow with an agentic step. The support system keeps classification, drafting and the final check fixed, and hands the "investigate" step to a small agent with six read-only tools and a cap of six turns. It returns structured findings: which invoices, whether the charge is really a duplicate, which policy applies. Any refund then goes through a fixed tool that requires approval above a limit.
An agent with workflow guardrails. The loop is free, but every write passes a fixed gate, and the final output must pass a fixed check before it leaves (Giving an Agent Room to Work covers how to bound it).
My rule of thumb: give the model freedom to read, and keep writes behind fixed gates. And when an agent has solved a recurring kind of case, consider turning the path it found into code or a checklist that runs cheaply, keeping the agent for the cases that don't fit.
What to do
Sort a few hundred recent cases into the paths a workflow would have. Count how many fit none. That share, more than any slogan, tells you whether you need an agent.
Ask how the system finds out it's wrong. If a fast, automatic check exists, an agent has something to push against. If every check takes a person's judgment, fixed review points are easier to run.
Price a wrong step. A high cost doesn't pick a side. It raises the value of whichever design is more reliable on your inputs, and it demands hard limits either way.
Count the repetitions. A stable task run thousands of times justifies building and testing a fixed path. A rare one costs more to design for than to run.
Then run both on your own data. A couple of hundred real requests, counting correct, escalated and silently wrong outcomes, will settle what no argument can.
And if you manage a team building one of these, four questions test their thinking. What share of last month's requests fit none of our paths, and what happened to them? How do we find out the system is wrong, and how fast? What's the most it can do without a person, and who set that limit? What did one run cost last week, including the expensive ones?
Check yourself
Why does a ten-step chain at 97 percent per step fail so often?
Per-step accuracy multiplies, so the whole chain is right only about 74 percent of the time.
When does an agent beat a workflow?
When it can catch and repair a good share of its own errors because the world pushes back, or when many real requests fall outside the designed paths.
What does a good hybrid look like?
A workflow with an agentic step, or an agent whose writes pass fixed gates. Give the model freedom to read and keep writes behind fixed gates.
Notes, sources and method
For readers who want to check the numbers and the worked comparison behind Workflows or Agents?. Sources were checked on 11 and 18 October 2026.
Key numbers
| Number in the article | Exact value and context | Source |
|---|---|---|
| Ten steps at 97% → about 74%; five → 86%; twenty → 54% | 0.97^10 = 73.7%; 0.97^5 = 85.9%; 0.97^20 = 54.4% (independent errors) | Computed |
| Eight-step agent, no recovery → 78% | 0.97^8 = 78.4% | Computed |
| Break-even recovery about 37%; 86% at 40%; 91% at 60% | r = 0.371; 86.5%; 90.8%. A longer agent needs more: about 50% at ten turns, 66% at fifteen | Computed (see Method) |
| About 3 cents vs about 12 cents per request; about 5 cents cached (1.6×) | Workflow reads 7,500 and writes 1,500 tokens ($0.030); agent reads 49,200 and writes 2,000 ($0.118, 3.95×); with caching $0.047 (1.57×) | Computed at Sonnet 5.5 list prices, Oct 2026: $2 / $10 per million input / output tokens; cache read $0.10, cache write $2.50 [9] |
| $4 per escalated case; $25 per wrong reply | Illustrative | Assumption |
| $2.08 vs $2.16 per request | Total expected cost on in-path requests, agent at 40% recovery; model bill 1.4% and 5.5% of these | Computed |
| A few percent off-path tips it | Crossover at 2.1% (r = 0.4); 18% (r = 0.2); 35% (r = 0); none at r = 0.6, where the agent is cheaper throughout | Computed |
| $30,000 vs $118,000 per million requests | Model spend only, uncached agent ($118,400) | Computed |
| Agents use several times a chat's tokens | About 4× (agents) and 15× (multi-agent systems) the tokens of a chat; multi-agent systems pay off only on high-value tasks | Anthropic, 13 Jun 2025 [4] (different task; magnitude check only) |
| τ-bench, pass^k; under half of tasks, under a quarter on all eight retail trials | Function-calling agents of 2024 | Yao et al., 17 Jun 2024 [6] |
| 75% per run → 42% for three in a row | 0.75³ = 0.42 | Anthropic, 9 Jan 2026 [5] |
| METR: 50% horizon doubling about every seven months since 2019; 80% horizon four to six times shorter | Self-contained software tasks; METR's page, last updated September 2026, warns that results above 16 hours are unreliable | Kwa et al. [7]; METR [8] |
| Anthropic: simplest design that works; OpenAI: grow in steps | Anthropic's advice is often a single model call with retrieval and examples. OpenAI names three conditions that make an agent a candidate: judgment with exceptions (refund approval), rule sets too unwieldy to maintain, heavy reliance on unstructured text | [1], [2] |
Definitions and patterns
The workflow/agent distinction comes from Anthropic's December 2024 post Building effective agents [1]; LangGraph's documentation draws the same line [3]. OpenAI's April 2025 guide uses "workflow" for the sequence of steps a goal requires, whoever runs it, and calls a system an agent when the model manages that execution [2]. When someone says a product "uses agents", ask which decisions the model makes and which the code makes.
Anthropic names five workflow patterns along the line from code in control to model in control:
| Pattern | Who decides what | In the support example |
|---|---|---|
| Prompt chaining | Code fixes the sequence; each call feeds the next, with code checks between | Classify, extract, draft, review |
| Routing | A model labels the input; code sends it down the matching branch | Billing and shipping have separate flows |
| Parallelization | Code runs calls at once, on independent pieces (sectioning) or repeats of one task (voting) | Three drafts compared before a refund |
| Orchestrator-workers | A model splits the task into subtasks chosen at run time; code owns the loop | A message with four questions becomes four lookups |
| Evaluator-optimizer | One call generates, another critiques; code repeats until a stopping rule is met | Draft, review against a rubric, revise |
Material dropped from the article body
- Four ways an agent fails. It loops (repeats a failing call), drifts (a long session fills its context and an early constraint stops carrying weight), wanders (does more than asked because each step looked reasonable) and overreaches (uses a tool for something nobody intended). Counterparts in code: step caps and budgets; restated constraints or a fresh context; narrow tool descriptions; permissions, approvals and reversibility. Anthropic mentions iteration limits and sandbox testing [1]; OpenAI failure thresholds that hand control to a person and approval for high-risk actions [2]. Retries of failed tool calls are covered in When the Tool Call Fails Halfway.
- Workflows fail quietly but repeatably. Input that fits no branch goes down the nearest one; a format change upstream or a language the router never saw produces normal-looking wrong output. The failure repeats, so once found it can be traced to a step, fixed and added to a test. Workflows also accumulate branches until they become the unwieldy rule set OpenAI describes.
- Outcome grading. Anthropic's January 2026 guidance recommends grading what an agent produced (the final state) ahead of the path, because checking a rigid sequence is brittle when agents find valid routes designers didn't foresee; transcripts supplement the outcome grade [5]. Anthropic's engineers also describe their research agents as non-deterministic between runs [4].
- Removing scaffolding on each model upgrade is the habit described in A Product That Improves With the Model. A workflow built for it passes plain structured data between stages and tests each stage alone, so merging two is an experiment with a clear answer.
- The bitter lesson. Hand-built chains are a textbook case of Sutton's bitter lesson, in which methods that encode human cleverness lose over time to general methods that use more computation [10].
- Cost trend. If token prices and caching keep falling, the cost gap narrows, though total spend can still rise with use (Where the AI Money Goes).
- The decision-guide figure (
a46-decision-guide) was dropped from the article in the 19 Oct 2026 rewrite; its four questions are in the article's "What to do". The SVG is kept.
Method
The comparison is a small probability and cost model in Python. Inputs: the workflow makes five model calls (classify, extract, pick the policy, draft, review), each reading about 1,500 tokens and writing 300, each right 97% of the time independently. The agent takes eight turns; each re-reads the whole context (3,000 tokens at first, growing by 900 per turn) and writes 250 tokens, each step right 97% of the time. Both end with the same deterministic check, which catches half of wrong drafts and sends them to a person. The agent hits its step cap on 4% of in-path and 8% of off-path requests (handed to a person). For off-path requests, the workflow's classifier forces 30% into a wrong branch (the rest reach "other" and go to a person), and the agent is right 95% of the time per step. Recovery share r ranges from 0 to 0.6, with 0.4 as the reference. Each outcome is correct, sent to a person, or wrong and silent, and the three sum to one at every off-path share. None of the inputs was measured on a real support operation.
Break-even recovery. With p = 0.97 per step, the five-step workflow is right with probability p⁵ = 0.8587. An agent that recovers a share r of its errors is right at each step with probability 1 − (1 − p)(1 − r), so right overall with probability (1 − 0.03(1 − r))⁸. Setting that equal to 0.8587 gives 1 − 0.03(1 − r) = 0.8587^(1/8) = 0.98114, so r = 0.371.
Sensitivity.
| Agent recovers | Cost per request with no off-path requests | Agent cheaper above this off-path share |
|---|---|---|
| none | workflow $2.08, agent $3.29 | about 35% |
| 20% | workflow $2.08, agent $2.74 | about 18% |
| 40% | workflow $2.08, agent $2.16 | about 2% |
| 60% | workflow $2.08, agent $1.56 | every share |
At 40% recovery, a classifier that misroutes 10% of unfamiliar requests instead of 30% moves the crossing to about 5%; with no recovery it moves from 35% to almost 90%. Give the workflow a validate-and-retry step that fixes 40% of its own errors and the crossing against a 40%-recovery agent rises to about 19%. Other runs varied the agent's off-path step success (90% to 97%), the cost of a wrong reply ($5 gives a 6.5% crossover; at $100 the agent is cheaper at every share) and the workflow's per-step success (98% and 99% move the crossover to 3.5% and 4.9%). If per-step reliability is far above 97%, compounding mostly disappears: at 99.5% a five-step chain is right 97.5% of the time.
The reference values were revised once after a first run (recovery 0.5 to 0.4, misrouting 60% to 30%); at the first values the agent was cheaper at every off-path share. At 0.4 recovery the two designs are near a tie on ordinary requests by construction, so a small off-path share tips the comparison.
"""A46 worked comparison: fixed workflow vs agent. Illustrative; every input is an assumption stated in the article.
Run: python3 -I A46.model.py (prints all numbers quoted in the article; writes A46.model.out.json)"""
import json
A = dict(
P_STEP=0.97, # per-step success of a model step, in-distribution, both designs
N_WF=5, # model calls in the workflow (classify, extract, pick policy, draft, review)
N_AG=8, # average loop iterations of the agent
R=0.4, # share of its own step errors the agent catches and fixes because a tool or check reports them
C=0.5, # share of wrong drafts the final deterministic validator catches (both designs): routed to a person
FORCE=0.3, # workflow: share of off-path requests the classifier forces into a wrong branch (the rest reach 'other' -> person)
P_OFF_AGENT=0.95, # agent per-step success on unfamiliar requests
STALL_ON=0.04, STALL_OFF=0.08, # agent hits its step cap and hands off to a person
PRICE_IN=2.0, PRICE_OUT=10.0, PRICE_CACHE=0.10, PRICE_CACHE_WRITE=2.50, # $ per million tokens (Sonnet 5.5 list, Oct 2026)
HUMAN_ESC=4.0, # $ per case a person has to handle (about 6 minutes of loaded labor)
HUMAN_WRONG=25.0, # $ per wrong reply that reaches a customer
)
def run(a, verbose=False):
wf_in, wf_out = a['N_WF']*1500, a['N_WF']*300
ag_in = sum(3000 + 900*k for k in range(a['N_AG'])); ag_out = a['N_AG']*250
def cost(i,o,cached=0.0):
return (i*((1-cached)*a['PRICE_IN'] + cached*a['PRICE_CACHE']) + o*a['PRICE_OUT'])/1e6
c_wf, c_ag = cost(wf_in,wf_out), cost(ag_in,ag_out)
# with prompt caching: each iteration's new tokens are written to the cache once, later iterations read the prefix
writes = 3000 + 900*(a['N_AG']-1)
reads = sum(3000 + 900*(k-1) for k in range(1, a['N_AG']))
c_agc = (writes*a['PRICE_CACHE_WRITE'] + reads*a['PRICE_CACHE'] + ag_out*a['PRICE_OUT'])/1e6
ok_wf = a['P_STEP']**a['N_WF']
u_on = (1-a['P_STEP'])*(1-a['R']); u_off = (1-a['P_OFF_AGENT'])*(1-a['R'])
ok_ag_on = (1-u_on)**a['N_AG']; ok_ag_off = (1-u_off)**a['N_AG']
def outcomes(f):
wrong_in = (1-f)*(1-ok_wf)
wf = dict(correct=(1-f)*ok_wf,
silent=wrong_in*(1-a['C']) + f*a['FORCE']*(1-a['C']),
esc=wrong_in*a['C'] + f*a['FORCE']*a['C'] + f*(1-a['FORCE']))
s = (1-f)*a['STALL_ON'] + f*a['STALL_OFF']
p = (1-f)*ok_ag_on + f*ok_ag_off
wrong = (1-s)*(1-p)
ag = dict(correct=(1-s)*p, silent=wrong*(1-a['C']), esc=s + wrong*a['C'])
for d,cm in ((wf,c_wf),(ag,c_ag)):
assert abs(d['correct']+d['silent']+d['esc']-1)<1e-9
d['model_cost']=cm
d['human_esc_cost']=d['esc']*a['HUMAN_ESC']; d['wrong_cost']=d['silent']*a['HUMAN_WRONG']
d['total']=cm+d['human_esc_cost']+d['wrong_cost']
return dict(f=f, wf=wf, ag=ag)
return dict(tokens=dict(wf=(wf_in,wf_out), ag=(ag_in,ag_out)), cost=dict(wf=c_wf, ag=c_ag, ag_cached=c_agc),
ok_wf=ok_wf, ok_ag_on=ok_ag_on, ok_ag_off=ok_ag_off, outcomes=outcomes)
def crossover(out, lo=0.0, hi=1.0):
g = lambda f: out(f)['wf']['total']-out(f)['ag']['total']
if g(lo)*g(hi) > 0: return None
for _ in range(60):
m=(lo+hi)/2
if g(lo)*g(m) <= 0: hi=m
else: lo=m
return (lo+hi)/2
if __name__ == '__main__':
r = run(A); out = r['outcomes']
print("tokens", r['tokens']); print("cost/run", {k: round(v,4) for k,v in r['cost'].items()},
"ratio uncached %.2f cached %.2f" % (r['cost']['ag']/r['cost']['wf'], r['cost']['ag_cached']/r['cost']['wf']))
print("chain 0.97^n", {n: round(.97**n,4) for n in (1,3,5,8,10,20)})
print("chain 0.99^n", {n: round(.99**n,4) for n in (5,10,20)}, " 0.95^n", {n: round(.95**n,4) for n in (5,10,20)})
print("agent in-path ok %.4f off-path ok %.4f (r=0 -> %.4f)" % (r['ok_ag_on'], r['ok_ag_off'], .97**8))
step = r['ok_wf']**(1/8); print("break-even r for 8-step agent vs 5-step chain: %.3f" % (1-(1-step)/0.03))
for n in (6,8,10,12,15):
st = r['ok_wf']**(1/n); print(" n=%d r*=%.3f" % (n, 1-(1-st)/0.03))
rows=[]
for f in (0.0,0.05,0.10,0.20,0.30):
o = out(f); rows.append(o)
print("f=%.2f | WF correct %.3f person %.3f silent-wrong %.3f total $%.2f | AG correct %.3f person %.3f silent-wrong %.3f total $%.2f" % (
f,o['wf']['correct'],o['wf']['esc'],o['wf']['silent'],o['wf']['total'],o['ag']['correct'],o['ag']['esc'],o['ag']['silent'],o['ag']['total']))
print("crossover f (total cost equal):", crossover(out))
print("--- sensitivity on R (agent recovery share)")
for R in (0.0,0.2,0.4,0.6):
b = dict(A, R=R); ro = run(b); oo = ro['outcomes']
print(" R=%.1f in-path ok %.3f | f=0: WF $%.2f AG $%.2f | f=.10: WF $%.2f AG $%.2f | f=.30: WF $%.2f AG $%.2f | crossover %s" % (
R, ro['ok_ag_on'], oo(0)['wf']['total'], oo(0)['ag']['total'], oo(.1)['wf']['total'], oo(.1)['ag']['total'], oo(.3)['wf']['total'], oo(.3)['ag']['total'], crossover(oo)))
print("--- sensitivity on cost of a silent wrong reply")
for W in (5,25,100):
b = dict(A, HUMAN_WRONG=W); oo = run(b)['outcomes']
print(" wrong=$%d | f=0: WF $%.2f AG $%.2f | f=.10: WF $%.2f AG $%.2f | crossover %s" % (W, oo(0)['wf']['total'], oo(0)['ag']['total'], oo(.1)['wf']['total'], oo(.1)['ag']['total'], crossover(oo)))
print("--- sensitivity on workflow per-step success (agent unchanged)")
for p in (0.97,0.98,0.99):
b = dict(A, P_STEP=p, P_OFF_AGENT=0.95); oo = run(b)['outcomes']
print(" p=%.2f | f=0: WF $%.2f AG $%.2f | f=.10: WF $%.2f AG $%.2f | crossover %s" % (p, oo(0)['wf']['total'], oo(0)['ag']['total'], oo(.1)['wf']['total'], oo(.1)['ag']['total'], crossover(oo)))
print("--- volume: model cost only, per 1M requests: WF $%.0f AG $%.0f AG(cached) $%.0f" % (r['cost']['wf']*1e6, r['cost']['ag']*1e6, r['cost']['ag_cached']*1e6))
json.dump(rows, open('A46.model.out.json','w'), indent=1)
Output:
tokens {'wf': (7500, 1500), 'ag': (49200, 2000)}
cost/run {'wf': 0.03, 'ag': 0.1184, 'ag_cached': 0.0472} ratio uncached 3.95 cached 1.57
chain 0.97^n {1: 0.97, 3: 0.9127, 5: 0.8587, 8: 0.7837, 10: 0.7374, 20: 0.5438}
chain 0.99^n {5: 0.951, 10: 0.9044, 20: 0.8179} 0.95^n {5: 0.7738, 10: 0.5987, 20: 0.3585}
agent in-path ok 0.8648 off-path ok 0.7837 (r=0 -> 0.7837)
break-even r for 8-step agent vs 5-step chain: 0.371
n=6 r*=0.165
n=8 r*=0.371
n=10 r*=0.496
n=12 r*=0.580
n=15 r*=0.663
f=0.00 | WF correct 0.859 person 0.071 silent-wrong 0.071 total $2.08 | AG correct 0.830 person 0.105 silent-wrong 0.065 total $2.16
f=0.05 | WF correct 0.816 person 0.110 silent-wrong 0.075 total $2.33 | AG correct 0.825 person 0.109 silent-wrong 0.067 total $2.22
f=0.10 | WF correct 0.773 person 0.149 silent-wrong 0.079 total $2.59 | AG correct 0.819 person 0.113 silent-wrong 0.069 total $2.28
f=0.20 | WF correct 0.687 person 0.227 silent-wrong 0.087 total $3.10 | AG correct 0.808 person 0.120 silent-wrong 0.072 total $2.40
f=0.30 | WF correct 0.601 person 0.304 silent-wrong 0.094 total $3.61 | AG correct 0.797 person 0.128 silent-wrong 0.076 total $2.52
crossover f (total cost equal): 0.02123740534270975
--- sensitivity on R (agent recovery share)
R=0.0 in-path ok 0.784 | f=0: WF $2.08 AG $3.29 | f=.10: WF $2.59 AG $3.46 | f=.30: WF $3.61 AG $3.80 | crossover 0.35422072960463535
R=0.2 in-path ok 0.823 | f=0: WF $2.08 AG $2.74 | f=.10: WF $2.59 AG $2.88 | f=.30: WF $3.61 AG $3.17 | crossover 0.18118698557353002
R=0.4 in-path ok 0.865 | f=0: WF $2.08 AG $2.16 | f=.10: WF $2.59 AG $2.28 | f=.30: WF $3.61 AG $2.52 | crossover 0.02123740534270975
R=0.6 in-path ok 0.908 | f=0: WF $2.08 AG $1.56 | f=.10: WF $2.59 AG $1.65 | f=.30: WF $3.61 AG $1.83 | crossover None
--- sensitivity on cost of a silent wrong reply
wrong=$5 | f=0: WF $0.67 AG $0.86 | f=.10: WF $1.02 AG $0.91 | crossover 0.06501496956512309
wrong=$25 | f=0: WF $2.08 AG $2.16 | f=.10: WF $2.59 AG $2.28 | crossover 0.02123740534270975
wrong=$100 | f=0: WF $7.38 AG $7.03 | f=.10: WF $8.48 AG $7.42 | crossover None
--- sensitivity on workflow per-step success (agent unchanged)
p=0.97 | f=0: WF $2.08 AG $2.16 | f=.10: WF $2.59 AG $2.28 | crossover 0.02123740534270975
p=0.98 | f=0: WF $1.42 AG $1.56 | f=.10: WF $2.00 AG $1.74 | crossover 0.03485563849827761
p=0.99 | f=0: WF $0.74 AG $0.93 | f=.10: WF $1.38 AG $1.18 | crossover 0.048554400591546254
--- volume: model cost only, per 1M requests: WF $30000 AG $118400 AG(cached) $47240
Caveats and open questions
- The crossover comes from the assumptions (independence, the recovery share, the misrouting rate, a workflow with no retries) and not from any property of agents. The model shows which quantities matter; only your own data can supply their values. Run both designs on a couple of hundred real requests and count correct, escalated and silently wrong outcomes.
- Wrong actions are not priced. The model prices wrong replies. A wrong refund or a message to the wrong person is a different cost, and only the agent holds tools that make it possible (Reversible Work and Irreversible Effects).
- Independence is unrealistic. Correlated errors would hurt chains more than agents that can re-plan; not modeled.
- What would change the conclusion: per-step reliability far above 97%; agent results stable from run to run (pass^k near single-run success); a measured recovery share below break-even for your step count (the agent then loses on ordinary requests and can only win where many requests leave the designed paths); falling token prices.
- That agents help least where nothing pushes back is an inference from the mechanism, not a measured result.
- The τ-bench figures are from 2024 models; only the abstract was read. METR's tasks are self-contained software work and say little about messy support queues. OpenAI's guide date comes from PDF metadata.
Sources
- Anthropic (Erik S. and Barry Zhang). Building effective agents. 19 Dec 2024. https://www.anthropic.com/engineering/building-effective-agents — workflow and agent definitions; the five patterns; simplest-solution advice; stopping conditions and sandbox testing; cost and compounding-error trade-off. The page notes that much of the tooling it describes has changed since publication.
- OpenAI. A practical guide to building agents. April 2025 (PDF metadata, 7 Apr 2025). https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf — its use of "workflow" and "agent"; three conditions favoring an agent; incremental approach; failure thresholds and high-risk actions.
- LangChain. Workflows and agents (LangGraph documentation). Undated; read 11 Oct 2026. https://docs.langchain.com/oss/python/langgraph/workflows-agents — the same split and patterns (it writes "orchestrator-worker").
- Anthropic. How we built our multi-agent research system. 13 Jun 2025. https://www.anthropic.com/engineering/multi-agent-research-system — about 4 times (agents) and 15 times (multi-agent) the tokens of a chat; viable only for high-value tasks; non-determinism between runs.
- Anthropic. Demystifying evals for AI agents. 9 Jan 2026. https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents — outcome versus transcript grading; brittle step checks; pass@k and pass^k, 75% to 42% example.
- Yao, Shinn, Razavi, Narasimhan. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045, 17 Jun 2024. https://arxiv.org/abs/2406.12045 — pass^k; agents of that time under 50% of tasks and under 25% on pass^8 in retail.
- Kwa et al. (METR). Measuring AI Ability to Complete Long Software Tasks. arXiv:2503.14499, v1 18 Mar 2025, v4 10 Jul 2026. https://arxiv.org/abs/2503.14499 and https://arxiv.org/html/2503.14499 — 50% time horizon doubling about every seven months since 2019; 80% horizon 4 to 6 times shorter.
- METR. Time horizons. Last updated 8 Sep 2026; read 11 Oct 2026. https://metr.org/time-horizons/ — horizon definition; task-suite and human-baseline caveats; no longer actively updated; results above 16 hours unreliable.
- Anthropic. Pricing. Read 11 Oct 2026. https://platform.claude.com/docs/en/about-claude/pricing — Sonnet 5.5 at $2 input and $10 output per million tokens; cache write $2.50, cache read $0.10.
- Sutton. The Bitter Lesson. 13 Mar 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html — the pattern named in the objection.