Systems around the model · Autonomy
Reversible Work and Irreversible Effects
How much an agent should be allowed to do alone depends less on how capable it is than on what one mistake costs, and that depends on whether the effect can be undone, offset or neither. Most of the design work is moving effects up that ladder, then spending human attention only on what stays at the bottom.
Everyone asks how good an AI agent is. The question that should decide how much it may do alone is different: what does one mistake cost?
Take two agents built on the same model, with the same error rate. One edits files in a repository under version control. When it gets something wrong, a person runs one command and the mistake is gone. The other pays suppliers. When it gets something wrong, the money may sit in an account nobody can reach. What differs is the price of being wrong once.
That price depends mostly on one property of the effect: whether it can be undone, whether it can be offset afterward, or neither. So here is the argument of this piece. Grant an agent autonomy per effect, by what an error in that effect would cost. Do most of the engineering moving effects to a cheaper class, and spend human approval only on what stays irreversible. And judge reversibility by what happens in the world, not by what the software can undo, because the two often disagree.
Three kinds of effect
Think of them as three rungs of a ladder, from the cheapest error to the most expensive.
Undo means the earlier state can be restored and nobody outside has acted on the change yet: a draft waiting in an outbox, a file under version control, a record edit that keeps its history, a payment scheduled for Friday that hasn't left the bank. The cost of an error is noticing it and pressing the button.
Compensate means the effect stays, but a new action can offset its consequence. A refund leaves the charge in place, and the customer sees both lines on the statement. A follow-up email ("please disregard the figure in my last message") arrives after the first one was read. This is a compensating action (When the Tool Call Fails Halfway covers running them reliably), and it always costs something beyond the money: staff time, a second message, a customer who now checks your invoices more carefully.
Neither means nothing restores the state or offsets the consequence. A message a customer has read. Personal data that reached the wrong person. A payment to a fraudster's account over rails that settle in seconds. A filing with a regulator. The error is paid in full.
Organizations already think this way about people. In his 2015 shareholder letter Jeff Bezos separated hard-to-reverse decisions, one-way doors that deserve slow, careful process, from two-way doors that people should make quickly, and warned that growing organizations apply the heavy process to everything and pay in slowness. The same warning applies to an agent with an approval on every action.
The analogy breaks in two places, and both matter. Bezos was describing a few big decisions weighed by people at the time. An agent takes thousands of small actions a day, so the classification has to be made in advance, per tool and per argument, by whoever builds the software around it. And a door is reversible or not for good, while an effect is reversible only until someone acts on it.
Reversible in the system, final in the world
That last point is where agents cause their most expensive surprises.
Email shows it cleanly. Gmail's undo send gives you up to 30 seconds to cancel. It works because the message hasn't left yet: the "undo" is really a short delay before the effect. Outlook's message recall tries to reach into the recipient's mailbox afterward, and works only if both people are in the same Microsoft 365 organization and the message is still unopened. The first design moved the effect to a safer class before it happened. The second tries to climb back once it has, and a read message stays read.
Now the case that matters most for business agents. An email asks the agent to update a supplier's bank details. The agent makes the change through the ERP's normal screen. In the system, the edit is perfectly reversible: one click restores the old account number. But the weekly payment run reads that field, and three days later €18,400 goes to the new account. The record can still be put back. The money can't.
An effect is reversible until something reads it and acts: the payment run, the customer who forwards the email, the colleague who builds a report on yesterday's figures. Just after that moment, recovery may still be possible with luck and speed. The FBI's internet crime unit says that in 2024 its recovery team acted on cases worth $848.4 million, mostly business email fraud, and froze about two-thirds of it. Those were cases reported fast enough to act on. Set against the $16.6 billion in total losses reported that year, the frozen amount is about 3 percent.
An effect is reversible until someone acts on it, and the system's undo button doesn't know when that happened.
So a bank-detail change is a final effect dressed as a reversible one. Classify an action by its chain of consequences, which can run well beyond the system it touches. The practical test is one question: what reads this next?
Move effects up before you decide who approves them
If the cost of an error depends on the class, the most productive question about a risky action is how to make it cheaper to get wrong. A handful of moves do most of the work.
Stage, then commit. Split the action into a reversible preparation and a small, separate commit: draft, then send; prepare the payment batch, then release it. Only the commit needs attention. OpenAI's Operator, a browsing agent released in January 2025, works this way for risky actions: it drafts the email, then asks whether to send it.
Add a delay. Put time between decision and effect, so canceling is cheap. Undo send is a 30-second version. European payment law has a longer one: a payment set for a future date can be revoked until the end of the previous business day. A three-day hold on the first payment to changed bank details applies the same idea on purpose. The price is waiting. Instant payments show the opposite extreme: funds arrive within ten seconds, any hour of any day, which leaves no window at all, so protection has to come before the payment.
Expose less at once. Try the change on one customer before all of them, one batch of reminders before the full list. Engineers call this a canary release. Caps on amounts, recipients and payments per hour do the same job: they turn an unbounded loss into a bounded one.
Verify through a different channel. Check the facts the effect depends on somewhere the agent didn't get them. Since October 2025, payment providers in the euro area must warn the payer when the name and account number don't match. For a bank-detail change, a call to the supplier on a number already on file does the same job.
Keep two people, or a person and a rule. No single actor should both start and approve a sensitive action, and the second check shouldn't share the first one's blind spots. The same model asked again with the same context is closer to asking the agent twice.
A worked example: an agent that pays suppliers
Put numbers on one process. A company handles 1,000 supplier invoices a month. An agent codes the invoices, answers supplier emails, prepares and releases payments, and handles the occasional request to change bank details. In a month that's about 2,400 actions. All the figures below are illustrative assumptions, chosen to be plausible rather than measured.
Most actions are cheap to get wrong: a miscoded invoice costs a few euros; a wrong payment to a regular supplier can usually be recovered from the next invoice. Two kinds are expensive. A payment to an account you've never paid before has no such channel back. And a fraudulent bank-detail change, which we price at €20,000 of diverted payments before anyone notices, is the worst of all. (The FBI counted about 21,000 business email fraud complaints in 2024, with $2.77 billion in reported losses, so the assumption is modest.)
Without any review, the expected loss is €8,575 a month. About 87 percent of it sits in just 115 actions: the bank-detail changes and the payments to new or changed accounts.
Now compare designs, assuming reviewers catch 90 percent of the errors they look at and costing their time at €60 an hour.
- Approve every action. All of them go to a person.
- Approve nothing. The agent acts alone, and errors surface later.
- Approve every payment. The intuitive design: payments can't be undone, so gate them all. Bank-detail changes are reversible record edits, so they go through.
- Approve what is final in the world. Gate bank-detail changes and payments to new or recently changed accounts, 115 actions. Let routine payments to verified accounts run under limits.
Guess first
The last design reviews about one action in twenty compared with approving everything. How much of the protection do you think it keeps?
Approving everything brings expected loss down to €858 a month, for 41 hours of review. Approving what is final in the world loses €1,825 for under four hours. It keeps 87.5 percent of the protection with 9 percent of the review time, and counting review as a cost, it's the cheapest design: €2,050 a month against €3,333.
The instructive one is "approve every payment". It spends 25 hours a month approving payments, nearly all of them routine, and still loses €6,618, barely better than no approvals at all. The most expensive error enters through an action it classified as reversible. A payment approver who sees a familiar supplier name, a matching invoice and an account number has no way to spot that the account changed last Tuesday.
The approval belongs at the step where irreversibility enters, which is often earlier than the step where money leaves.
The conclusion holds up when the assumptions move. If every reviewer catches only half the errors, gating what's final is still cheapest. If the model gets twice as good, the gate still pays for itself many times over; it only stops paying when errors on those 115 actions fall about thirtyfold, and one of those rows is set by how hard fraudsters try, not just by how good the model is (Giving an Agent Room to Work). The one case where it flips is a process whose risk is spread evenly over many cheap, compensable actions. There, acting freely and sampling afterward wins. Check where your expected loss actually sits before gating anything.
The classification belongs in code
The example assumes the software knows which class each action belongs to. That knowledge has to live in code, next to the tools, not in the prompt. An instruction saying "ask before anything irreversible" asks the model to recognize irreversibility in the moment, under pressure to finish, possibly after the instruction has scrolled out of what it can see. Same Model, Different Results calls such sentences wishes, and contrasts them with walls, controls that hold whatever the model concludes.
OpenAI's own numbers show both the value and the limit of letting the model decide when to ask. In its Operator testing, mitigations led by training the model to ask before risky actions cut that risk by about 90 percent. On the tasks where confirmation was needed, it asked 92 percent of the time, which also means it skipped 8 percent. A check in the tool layer has no skip rate. It fires on every call it covers.
For a business agent, the classification is a short table kept next to the tool definitions: each tool's class, the argument values that change it, the limits that apply, and who approves what remains. Arguments matter as much as tools. The same send_email is a draft when addressed to the outbox, compensable when sent to a colleague, and final when sent to a customer with a figure in it. The same release_payment is compensable to a supplier you pay monthly and final to an account first seen on Tuesday. The software knows those facts at the moment of the call, from the recipient list and the supplier records, so classifying is a lookup.
Two defaults keep the table honest. An effect nobody classified counts as final until someone says otherwise. A model under evaluation once submitted an invented tip through a police department's web form: its instructions barred destructive submissions, nothing covered a tip form, and nothing beneath the model asked (When Test Agents Touch the Real World). And every reversible write gets one more question: what reads it next? That question is what catches the bank-detail change.
Why this isn't bureaucracy
The strongest objection goes like this: models are improving fast, every classification table is one more thing to maintain, and Bezos himself warned against treating every decision as a one-way door. A capable model told to be careful with payments will usually be careful.
Part of that is right: an approval queue people click through unread is worse than none. But the classification I'm arguing for removes far more approvals than it adds. In the example it cuts reviewed items from about 2,400 to 115. The expensive bureaucracy is reviewing everything, which is exactly what this rejects.
My view is that classification is what buys an agent its freedom. Where effects can be undone or offset and every action is logged, the worst case is bounded and visible, and the agent can act freely. What can't be made reversible (a statement to a customer about their account, a disclosure to a regulator, a large payment to a new counterparty) gets a person who decides, sees the evidence, and is named. That last part does a job the arithmetic can't measure: every audit, contract and court will ask who decided (Who Is Responsible When an Agent Acts).
One more thing to keep reversible: the decision to automate. A switch that sends a category of work back to people only works if the people still have the procedure, the access and the time. Otherwise switching the agent off simply stops the work.
What to do
List your agent's effects and ask of each: what reads this next? Classify by the answer, not by what the software can undo.
Move effects up before adding approvals. Stage then commit, add delays and holds on first payments to changed accounts, cap amounts, verify through a second channel, start with one customer.
Put approvals where irreversibility enters, which is often a record edit days before the money moves, and show the approver what changed and when.
Keep the classification in code, per tool and per argument, and treat anything unclassified as final.
Find where your expected loss sits. If it's concentrated, gate the few actions that hold it. If it's spread thin over cheap, compensable work, let the agent act and sample afterward.
Check yourself
Why was the bank-detail change a final effect dressed as a reversible one?
The record could be restored with one click, but the weekly payment run read the field and sent the money to the new account. The money could not be put back.
Why did approving every payment lose almost as much as approving nothing?
It spent review on routine payments, while the costly error entered through the bank-detail edit, which looked reversible. An approver seeing a familiar supplier and a matching invoice cannot tell the account changed.
Why does the classification belong in code and not in the prompt?
A prompt asks the model to recognize irreversibility in the moment, and it sometimes skips the question. A check in the tool layer fires on every call it covers.
Notes, sources and method
For readers who want to check the numbers and the worked example behind Reversible Work and Irreversible Effects. Sources were checked on 11 and 18 October 2026.
Key numbers
| Number in the article | Exact value and context | Source and date |
|---|---|---|
| One-way and two-way doors | Type 1 decisions "methodically, carefully, slowly"; Type 2 "made quickly by high judgment individuals or small groups"; large organizations tend to use the heavy process "on most decisions, including many Type 2 decisions", paying in slowness, unthoughtful risk aversion and failure to experiment | Bezos, 2015 letter to shareholders, filed 5 Apr 2016 [1] |
| Gmail undo send, up to 30 seconds | Cancellation period of 5, 10, 20 or 30 seconds | Gmail Help, accessed 11 Oct 2026 [2] |
| Outlook recall conditions | Both people in the same Microsoft 365 organization, and the message unopened | Microsoft Support, accessed 11 Oct 2026 [3] |
| €18,400 to the new account; payment run on day 3 | Illustrative | Worked example |
| FBI recovery: $848.4 million, about two-thirds frozen | Recovery Asset Team acted on 3,020 complaints, mostly business email compromise; $469.1 million (domestic) plus $92.5 million (international) frozen = $561.6 million, 66% | FBI IC3, Internet Crime Report 2024 [11] |
| About 3% of total losses | $561.6 million / $16.6 billion total reported losses ≈ 3.4% | Computed from [11] |
| About 21,000 business email fraud complaints, $2.77 billion | 21,442 complaints; $2,770,151,146 reported losses; average about $129,000 per complaint | [11] |
| Operator, January 2025; draft then ask | Research preview; drafts the email and asks before sending | OpenAI, Operator System Card, 23 Jan 2025 [7] |
| About 90% risk reduction; 92% asked, 8% skipped | Unmitigated model: 13 errors on 100 representative tasks, 8 reversible within minutes, 5 irreversible or possibly severe (including an email to the wrong recipient). Mitigations, primarily confirmations, cut risk by about 90%. Confirmation recall 92% on 607 tasks in the risky categories | [7] |
| Future-dated payment revocable until the end of the previous business day | PSD2 Article 80: an order generally can't be revoked once the payer's bank received it; orders for an agreed future day are revocable until the end of the preceding business day | PSD2 [8] |
| Instant payments within ten seconds, any hour | Funds available within ten seconds, 24/7/365 | ECB [9] |
| Verification of payee since October 2025 | Euro area from 9 October 2025; warns on name and account mismatch | ECB [9] |
| Canary | Partial, time-limited deployment of a change with evaluation; a defect's cost proportional to the traffic exposed | Google SRE Workbook [10] |
| Worked example results | See the tables below | Computed |
Material dropped from the article body
- Two hidden conditions in "undo": somebody kept the prior state, and nobody has relied on the new one.
- Disclosure. Deleting a leaked file removes your copy and nobody else's. GDPR Article 17(2) asks a controller who made personal data public to take "reasonable steps", given technology and cost, to inform other controllers of an erasure request; it can oblige you to try, not make the copies disappear [4].
- Claude Code. Claude Code, Anthropic's coding agent, checkpoints the files its editing tools change and can rewind them; files changed by shell commands are not tracked, and checkpoints don't replace version control (documentation as of October 2026) [5]. Whether an action can be rewound depends on which tool path performed it. Its permission system runs read-only tools without approval inside the working directory, asks by default for file edits and most shell commands, applies deny rules over allow rules, and documents the mode that skips prompts as suitable only for isolated environments such as containers or virtual machines: the sandbox moves every effect up the ladder first, and only then is approval dropped [6].
- PSD2 Article 88. When the account identifier given was wrong, the bank isn't liable and must only make reasonable efforts to recover the funds [8]. This is why a wrong payment to a never-paid account sits on the bottom rung.
- Dry runs, sandboxes and checkpoints as a move up the ladder. Shadow mode, described in Moving Without Breaking, is a dry run at the scale of a whole process. The price is maintaining the copy, and a dry run misses whatever the copy doesn't reproduce.
- Segregation of duties is the formal name for the two-person rule: no single actor both initiates and approves a sensitive action.
- The off switch. Moving Without Breaking argues for a switch that returns a category of work to the previous stage; it stops future effects only, and payments already sent stay sent. Why the Process Exists makes the related point that a removed step's function, including being the fallback, has to survive somewhere. Where review earns its cost is taken up in Human Review Where It Matters.
- The ladder figure (
a108-ladder) was dropped from the article in the 19 Oct 2026 rewrite; its SVG is kept. It lists the moves that push effects up: stage then commit, delay windows and holds, dry runs and sandboxes, checkpoints and history, canaries, value and rate limits, verification before sending, known recipients only.
Worked example: inputs
All inputs are illustrative assumptions, not measurements.
| Action | Rung | Per month | Error rate | Cost per error | Review time |
|---|---|---|---|---|---|
| Record edits (coding, status, holds) | Undo | 1,000 | 2% | €8 | 30 s |
| Supplier emails | Compensate | 400 | 1% | €60 | 1 min |
| Payments to verified, unchanged accounts | Compensate | 900 | 0.5% | €150 | 1.5 min |
| Payments to new or recently changed accounts | Neither | 100 | 0.5% | €3,000 | 1.5 min |
| Bank-detail changes | Undo in the system, neither in the world | 15 | 2% | €20,000 | 5 min |
Rung justifications: a wrong payment to a supplier paid every month is usually compensable (ask for it back or deduct it from the next invoice; the supplier has a relationship to protect). A wrong payment to a new account has no such channel (PSD2 Article 88). €20,000 stands for the payments diverted before a fraudulent bank-detail change is noticed.
Expected monthly loss without review: €160 (coding), €240 (emails), €675 (routine payments), €1,500 (new accounts), €6,000 (bank-detail changes); €8,575 in total, about 87% of it in 115 of 2,415 actions.
Worked example: results
Reviewers catch 90% of errors in gated actions; review time costs €60 an hour.
| Design | Gated actions | Expected loss | Review hours | Total incl. review |
|---|---|---|---|---|
| 1. Approve every write | 2,415 | €858 | 41.3 | €3,333 |
| 2. No approvals | 0 | €8,575 | 0 | €8,575 |
| 3a. Gate by system class (all payments) | 1,000 | €6,618 | 25.0 | €8,118 |
| 3b. Gate by world class (bank-detail changes, new-account payments) | 115 | €1,825 | 3.8 | €2,050 |
The article calls 3a "approve every payment" and 3b "approve what is final in the world". Design 3b keeps 87.5% of design 1's loss reduction with 9.1% of its review hours and 4.8% of its reviewed items. Design 3a's poor result depends on the payment approver not seeing that the account changed; a payment screen that flags recent changes would help, and that flag is itself a classification by what happened in the world.
Sensitivity.
- Reviewer fatigue. Design 1 asks one team to review about 2,400 items a month; automation bias makes 90% generous at that volume. At 70% for design 1 alone, its total rises from €3,333 to €5,048.
- Weak review everywhere. At 50% catch for all designs, 3b still costs least: €5,050, against €6,763 (design 1) and €8,575 (no approvals).
- A better model. Halving every error rate: design 2 loses €4,288; design 3b loses €913 plus €225 of review (€1,138). Gating the two bottom-rung rows prevents €6,750 a month at base rates for €225 of review, so it keeps paying until error rates on those rows fall about thirtyfold. The bank-detail row is adversarial: its error rate depends on how hard fraudsters try as much as on model quality.
- A smaller fraud cost. At €2,000 per accepted fraudulent change, 3b still has the lowest total (€1,510, against €2,793 for design 1 and €3,175 for no approvals).
- No concentration. Remove the bank-detail row and price new-account payments at €150: no approvals becomes the cheapest design (€1,150).
- What the model leaves out. Review is the only control modeled. A call-back to a known number lowers the bank-detail error rate, a hold on first payments to changed accounts turns part of the bottom rung into undo, and a cap bounds the cost per error.
Method
Expected loss per class is count × error rate × cost. A gated class keeps 10% of its expected loss; an ungated class keeps all of it. Review hours are the sum of count × review time over gated classes, costed at €60 per hour. Errors are independent; the fraud loss is attributed to the bank-change decision only (a gated payment to a changed account would be a second chance to catch it, not counted, which is conservative for design 3b). The FBI shares were computed from the report's published totals (561.6 / 848.4 ≈ 66%; 561.6 / 16,600 ≈ 3.4%; 2.77 billion / 21,442 ≈ $129,000).
Script (Python 3, standard library only):
# A108 worked example: expected monthly loss and review effort under four autonomy designs.
# All inputs are illustrative assumptions, not measurements.
rows = [ # id, label, rung, count/month, p_error, cost_per_error_eur, review_min
("edit", "Record edits (coding, status, holds)", "undo", 1000, 0.02, 8, 0.5),
("email", "Supplier emails (replies, remittances)", "compensate", 400, 0.01, 60, 1.0),
("pay_k", "Payments to verified, unchanged accounts", "compensate", 900, 0.005, 150, 1.5),
("pay_n", "Payments to new or recently changed accounts", "neither", 100, 0.005, 3000, 1.5),
("bank", "Supplier bank-detail changes", "undo in system, neither in world", 15, 0.02, 20000, 5.0),
]
CATCH = 0.9 # share of errors a reviewer catches when the action is gated
RATE = 60.0 # loaded cost of review time, EUR per hour
designs = {
"1 Approve every write": {"edit","email","pay_k","pay_n","bank"},
"2 No approvals": set(),
"3a Gate by system class (payments)": {"pay_k","pay_n"},
"3b Gate by world class": {"pay_n","bank"},
}
def run(catch=CATCH, scale=1.0, catch_d1=None):
out = {}
for d, gated in designs.items():
loss = mins = items = 0.0
for rid, _, _, n, p, c, t in rows:
el = n * p * scale * c
if rid in gated:
k = catch_d1 if (catch_d1 is not None and d.startswith("1")) else catch
loss += el * (1 - k); mins += n * t; items += n
else:
loss += el
out[d] = dict(loss=loss, hours=mins/60, items=items, review_eur=mins/60*RATE,
total=loss + mins/60*RATE)
return out
print("Expected loss per row with no review:")
tot = 0
for rid, lab, rung, n, p, c, t in rows:
el = n*p*c; tot += el
print(f" {lab:48s} {rung:34s} n={n:5d} p={p:.3f} c={c:6d} EL=EUR {el:8.1f} review={n*t/60:5.2f} h")
print(f" TOTAL EUR {tot:.1f}\n")
base = run()
for d, r in base.items():
print(f"{d:38s} loss EUR {r['loss']:8.1f} review {r['hours']:5.2f} h ({r['items']:.0f} items) review cost EUR {r['review_eur']:7.1f} total EUR {r['total']:8.1f}")
d1, d2, d3 = base["1 Approve every write"], base["2 No approvals"], base["3b Gate by world class"]
prev1 = d2["loss"] - d1["loss"]; prev3 = d2["loss"] - d3["loss"]
print(f"\n3b keeps {prev3/prev1:.1%} of design 1's loss reduction with {d3['hours']/d1['hours']:.1%} of its review hours and {d3['items']/d1['items']:.1%} of its reviewed items")
print(f"3b avoids {prev3/d2['loss']:.1%} of unreviewed expected loss")
print("\nSensitivity: design 1 reviewers catch 70% (fatigue), others 90%")
for d, r in run(catch_d1=0.7).items(): print(f" {d:38s} loss {r['loss']:8.1f} total {r['total']:8.1f}")
for s in (0.5, 0.1):
print(f"\nSensitivity: all error rates x{s}")
for d, r in run(scale=s).items(): print(f" {d:38s} loss {r['loss']:8.1f} total {r['total']:8.1f}")
gated_el = sum(n*p*c for rid,_,_,n,p,c,_ in rows if rid in designs["3b Gate by world class"])
gate_cost = base["3b Gate by world class"]["review_eur"]
print(f"\nBreak-even: gating rows saves {CATCH*gated_el:.0f} EUR/month at base rates for {gate_cost:.0f} EUR of review;"
f" gating stops paying when those error rates fall by more than {CATCH*gated_el/gate_cost:.1f}x")
print("\nSensitivity: all reviewers catch 50%")
for d, r in run(catch=0.5).items(): print(f" {d:38s} loss {r['loss']:8.1f} total {r['total']:8.1f}")
print("\nSensitivity: fraudulent bank change costs EUR 2,000 instead of 20,000")
rows[4] = rows[4][:5] + (2000,) + rows[4][6:]
for d, r in run().items(): print(f" {d:38s} loss {r['loss']:8.1f} total {r['total']:8.1f}")
print("\nSensitivity: no concentration (no bank-change row; new-account payments cost EUR 150)")
rows.pop(4); rows[3] = rows[3][:5] + (150,) + rows[3][6:]
for d, r in run().items(): print(f" {d:38s} loss {r['loss']:8.1f} total {r['total']:8.1f}")
Output:
Expected loss per row with no review:
Record edits (coding, status, holds) undo n= 1000 p=0.020 c= 8 EL=EUR 160.0 review= 8.33 h
Supplier emails (replies, remittances) compensate n= 400 p=0.010 c= 60 EL=EUR 240.0 review= 6.67 h
Payments to verified, unchanged accounts compensate n= 900 p=0.005 c= 150 EL=EUR 675.0 review=22.50 h
Payments to new or recently changed accounts neither n= 100 p=0.005 c= 3000 EL=EUR 1500.0 review= 2.50 h
Supplier bank-detail changes undo in system, neither in world n= 15 p=0.020 c= 20000 EL=EUR 6000.0 review= 1.25 h
TOTAL EUR 8575.0
1 Approve every write loss EUR 857.5 review 41.25 h (2415 items) review cost EUR 2475.0 total EUR 3332.5
2 No approvals loss EUR 8575.0 review 0.00 h (0 items) review cost EUR 0.0 total EUR 8575.0
3a Gate by system class (payments) loss EUR 6617.5 review 25.00 h (1000 items) review cost EUR 1500.0 total EUR 8117.5
3b Gate by world class loss EUR 1825.0 review 3.75 h (115 items) review cost EUR 225.0 total EUR 2050.0
3b keeps 87.5% of design 1's loss reduction with 9.1% of its review hours and 4.8% of its reviewed items
3b avoids 78.7% of unreviewed expected loss
Sensitivity: design 1 reviewers catch 70% (fatigue), others 90%
1 Approve every write loss 2572.5 total 5047.5
2 No approvals loss 8575.0 total 8575.0
3a Gate by system class (payments) loss 6617.5 total 8117.5
3b Gate by world class loss 1825.0 total 2050.0
Sensitivity: all error rates x0.5
1 Approve every write loss 428.7 total 2903.8
2 No approvals loss 4287.5 total 4287.5
3a Gate by system class (payments) loss 3308.8 total 4808.8
3b Gate by world class loss 912.5 total 1137.5
Sensitivity: all error rates x0.1
1 Approve every write loss 85.7 total 2560.8
2 No approvals loss 857.5 total 857.5
3a Gate by system class (payments) loss 661.8 total 2161.8
3b Gate by world class loss 182.5 total 407.5
Break-even: gating rows saves 6750 EUR/month at base rates for 225 EUR of review; gating stops paying when those error rates fall by more than 30.0x
Sensitivity: all reviewers catch 50%
1 Approve every write loss 4287.5 total 6762.5
2 No approvals loss 8575.0 total 8575.0
3a Gate by system class (payments) loss 7487.5 total 8987.5
3b Gate by world class loss 4825.0 total 5050.0
Sensitivity: fraudulent bank change costs EUR 2,000 instead of 20,000
1 Approve every write loss 317.5 total 2792.5
2 No approvals loss 3175.0 total 3175.0
3a Gate by system class (payments) loss 1217.5 total 2717.5
3b Gate by world class loss 1285.0 total 1510.0
Sensitivity: no concentration (no bank-change row; new-account payments cost EUR 150)
1 Approve every write loss 115.0 total 2515.0
2 No approvals loss 1150.0 total 1150.0
3a Gate by system class (payments) loss 475.0 total 1975.0
3b Gate by world class loss 1082.5 total 1232.5
Caveats and open questions
- What would change the conclusion. (1) A large, durable drop in error rates on bottom-rung actions that holds under adversarial pressure; in the example, approval stops paying at about a thirtyfold improvement on those rows. Production data on agent errors broken down by class, of the kind the Operator card reports for one model in one evaluation, would show how far a given process is from that point. (2) Infrastructure that moves effects up by default: verification of payee, holds on new beneficiaries, mail systems that delay external sends, APIs that accept a staged action and a separate commit. (3) A process whose expected loss is spread evenly over many cheap, compensable actions, where acting freely and sampling afterward wins.
- No production data on agent error rates by reversibility class exists beyond Operator's single evaluation; the worked example is illustrative.
- Whether concentrated review is more careful than blanket review is a hypothesis.
- PSD2 is cited as the law in force. Its replacement (the Payment Services Regulation and PSD3) was not checked and should be before publication. EPC rulebook specifics were not fetched.
- Microsoft's recall page has inconsistent statements for Outlook on the web; only the two conditions above are used.
Sources
- Jeff Bezos. 2015 Letter to Shareholders. Amazon.com, Inc., Exhibit 99.1 to Form 8-K filed with the SEC, April 2016. https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm — Type 1 and Type 2 decisions, one-way and two-way doors, heavy process applied to reversible decisions (filed 5 April 2016).
- Google. Undo sending your mail. Gmail Help, accessed 11 Oct 2026. https://support.google.com/mail/answer/2819488 — cancellation period of 5, 10, 20 or 30 seconds.
- Microsoft. Recall or replace an email message that you sent. Microsoft Support, accessed 11 Oct 2026. https://support.microsoft.com/en-us/office/recall-or-replace-an-email-message-that-you-sent-35027f88-d655-4554-b4f8-6c0729a723a0 — recall requires the same Microsoft 365 organization and an unopened message.
- Regulation (EU) 2016/679 (GDPR), Article 17. Text via gdpr-info.eu, accessed 11 Oct 2026. https://gdpr-info.eu/art-17-gdpr/ — right to erasure; "reasonable steps" to inform other controllers when data was made public.
- Anthropic. Checkpointing. Claude Code documentation, accessed 11 Oct 2026. https://code.claude.com/docs/en/checkpointing — file edits tracked and rewindable; shell-command changes not tracked; not a replacement for version control.
- Anthropic. Configure permissions. Claude Code documentation, accessed 11 Oct 2026. https://code.claude.com/docs/en/permissions — approval tiers by tool type, session-scoped approval for file edits, deny-ask-allow rule order.
- OpenAI. Operator System Card. 23 January 2025. https://cdn.openai.com/operator_system_card.pdf — 13 errors in 100 tasks (8 easily reversible, 5 irreversible or possibly severe), confirmations reducing risk by about 90%, 92% confirmation recall on 607 tasks, draft-then-confirm examples.
- Directive (EU) 2015/2366 (PSD2), Articles 78, 80 and 88. Text via legislation.gov.uk (EU directives as published), accessed 11 and 18 Oct 2026. https://www.legislation.gov.uk/eudr/2015/2366/article/80 and https://www.legislation.gov.uk/eudr/2015/2366/article/88 — irrevocability once received, revocation of future-dated orders until the preceding business day, incorrect identifiers and "reasonable efforts" to recover.
- European Central Bank. Instant payments and Instant Payments Regulation. Accessed 11 Oct 2026. https://www.ecb.europa.eu/paym/integration/retail/instant_payments/html/index.en.html and https://www.ecb.europa.eu/paym/integration/retail/instant_payments/html/instant_payments_regulation.en.html — funds available within ten seconds, 24/7/365; verification of payee from 9 October 2025 in the euro area.
- Google. Canarying Releases. The Site Reliability Workbook, chapter 16. https://sre.google/workbook/canarying-releases/ — definition of canarying as partial, time-limited deployment and evaluation; impact proportional to traffic exposed.
- Federal Bureau of Investigation. Internet Crime Report 2024. Internet Crime Complaint Center. https://www.ic3.gov/AnnualReport/Reports/2024_IC3Report.pdf — business email compromise complaints (21,442) and losses ($2,770,151,146); Recovery Asset Team: 3,020 complaints, $848.4 million attempted, $469.1 million (domestic) plus $92.5 million (international) frozen, 66% success rate; most kill-chain incidents it initiates are business email compromise.
- Hector Garcia-Molina and Kenneth Salem. Sagas. Proceedings of ACM SIGMOD 1987. Princeton technical report CS-TR-070-87: https://www.cs.princeton.edu/techreports/1987/070.pdf — compensating transactions as semantic, not literal, undo (scanned report, read 18 Oct 2026).