When the Tool Call Fails Halfway

Your agent asks a payment provider to charge a customer $49 for a subscription renewal. Five seconds pass. No answer comes back.

Did the customer pay?

The honest answer is that nobody knows. Maybe the request never arrived. Maybe the provider is halfway through. Maybe the charge went through and the reply got lost on the way back. From where the agent sits, all three look identical.

What the agent does next decides whether the customer pays $49 or $98. And this is where AI agents make an old problem worse, because the most natural thing for a language model to do after an error is to try again.

After a timeout the truth is "unknown", and a model is built to read it as an ordinary error and retry. Telling the model to be careful doesn't help, because the model is missing information that no instruction can supply. The protection has to live in the code beneath the model: a name for every intended action, a record of what was done kept outside the model, a way to ask the world what happened, and a plan for undoing steps that belong together. Distributed systems engineers have used these tools for decades. Agents need them more.

Six seconds, two charges

Look at the renewal in slow motion. The agent calls a tool named charge_customer. The tool forwards the request to the payment provider. The provider is under load and takes about six seconds. The harness, the ordinary software that runs the model and its tools, gives each tool call five.

Scroll sideways to see the whole figure

At 5.0 seconds the harness stops waiting. The charge hasn't happened yet. At 6.2 seconds it has, and the provider's reply arrives at a connection nobody is listening on anymore. Between those two moments the world changed and the harness learned nothing. So it reports the only thing it knows: an error, "timeout after 5s."

Now the model's training and instructions take over. The Model Context Protocol, a widely used open standard for connecting models to tools, gives a tool result a single error flag and says such errors carry feedback models can use "to self-correct and retry with adjusted parameters." For a date in the wrong format, that is exactly right. For a payment, it is a disaster. The model reads "error", does what errors invite, and calls charge_customer again. The provider sees a brand new request and charges again.

Payment companies have long known better. Stripe tells developers to treat a server error as indeterminate, because the operation may already have happened, and not to retry with a new request until they know. The flag that reaches the model has two values. The world has three states.

After a timeout, the honest answer to "did it happen?" is "unknown", and the code has to carry that third state instead of rounding it to failure.

Canceling doesn't rescue you either. A cancellation can arrive after the work is finished, and the server is allowed to ignore it. Canceling stops unfinished work. It cannot recall finished work.

Why an agent makes it worse

Distributed systems engineers have lived with lost replies forever. What changes with an agent is where the retry comes from and how much the caller knows.

The retry comes from the top. In normal software, a retry happens inside a loop a programmer wrote, often inside the payment library, which can attach the same identifier to each attempt so the provider recognizes the duplicate. When the model retries, it issues a new tool call. The tool code runs from the beginning, the library creates a fresh identifier, and the provider sees a new payment. Layers multiply: a library allowed two retries makes up to three requests per call, and a model that tries three times makes up to nine requests under three different identifiers. That can mean three charges.

The second attempt isn't identical. A model regenerates its arguments every time. amount: 4900 becomes amount: "49.00", a subject line gets reworded. A guard that spots duplicates by comparing arguments sees two different requests.

The model's account of what happened is just text. When a model tells a user "the payment went through", it is producing a plausible continuation of what it has read. If that includes an ambiguous timeout, "failed" and "succeeded" are both plausible, and the user reads whichever comes out as a fact.

The evidence can disappear. Long-running agents work within a context window, the limited amount of text a model can take in at once, and harnesses clear old tool results to make room. For file contents that's the right trade. For effects, it means an agent that sent forty payment reminders an hour ago may no longer have any trace of them, and when asked to make sure every overdue client got one, its only way to check is to send them again (Agents That Work for Hours). And processes crash and restart, taking whatever they held in memory with them.

Give every intended action a name

The core fix is an old one. An action is idempotent if doing it twice leaves the world as doing it once would. "Set the ticket status to closed" is idempotent. "Charge $49" is not.

An idempotency key turns the second kind into the first. The caller gives the intended action a name and sends it with every attempt. The receiver remembers which names it has already carried out. A second request with the same name gets back the saved result of the first, and nothing runs twice.

Your bank already works this way. It won't cash check #1042 twice, because the number travels with the payment and the bank records the numbers it has cleared. But the analogy also shows where agents go wrong. A fresh check with a new number is, to the bank, a new payment, and that is exactly what a model creates when it reissues a call.

So two details decide whether a key protects you.

Where the key comes from. A key works only if every attempt at the same intention carries it, across retries, crashes and restarts. That means building it from something stable, such as the task and step ("renewal 8812, step 3") or the business object ("invoice 2291, October renewal"). The harness creates the key when the step is first planned and writes it down before the first attempt. The model never chooses it, because a model asked for a key would invent a new one each time.

How long it lasts. Receivers don't remember forever. Stripe's original API keeps keys for at least 24 hours. An email provider that accepts keys keeps them 24 hours too. An agent that waits over a weekend for a human approval can outlive its key, and then a reused key simply runs as a new request.

Ask the world before trying again

When the key has expired, or the provider never accepted one, the safe move is reconciliation: ask the other system what happened, then decide. Stripe is explicit that once the window has passed with the outcome unknown, you should look the outcome up rather than retry with either the old key or a new one.

That only works if you can find your own effect afterward. So name every effect before you cause it: the step's key in the payment's metadata, your own identifier in an email's headers, your reference on the calendar event. A system that names its effects can ask about them later. One that lets the provider assign every identifier has to guess.

Timing matters too. In the timeline above, a lookup at 5.1 seconds would have found nothing, because the charge landed at 6.2. Waiting, checking again, or listening for the provider's notifications closes that gap.

Every question in this sequence (was it a read? did the request reach the server? is the key still alive? does a lookup find it?) has a factual answer the harness can determine in code. None needs the model's judgment, and the model is badly placed to give it anyway: it can't see the provider, and may no longer see the earlier attempt.

Keep a journal the model can't lose

Keys protect one call. A task is a sequence of calls, and the process running it can die between any two.

The answer is a journal: an append-only record, kept outside the model's context, written in a fixed order. Before acting, record the step: which tool, its arguments as first decided, its key. After acting, record the outcome. If the process crashes, run the task again from the top. Steps with a recorded outcome return it without calling anything. A step written down without an outcome is the unknown case: look it up by its key, or retry with the same key while it's still valid.

Scroll sideways to see the whole figure

This is the core of what engineers call durable execution, and platforms such as Temporal are built around it. Agents need one addition. The model's decision is itself unpredictable, so it has to be a journaled step too. If the harness records "call charge_card with these arguments" before executing it, a resumed task re-reads that decision instead of asking the model again, and the model never gets the chance to phrase the arguments differently. The model is consulted only for steps that haven't been decided yet, and what it sees about earlier steps comes from the journal, which no context clearing can touch.

On resume, an agent should re-read its decisions instead of making them again.

Effects that belong together

Some tasks are several effects that only make sense as a set: hold a flight, book a hotel, charge the card, email the itinerary. No single transaction spans an airline, a hotel chain and a payment provider. If the hotel step fails, nothing releases the flight hold by itself.

The standard answer dates from 1987, when two database researchers proposed splitting long transactions into steps, each paired with a compensating action that offsets it. They called the result a saga: either every step completes, or the completed ones are followed by their compensations in reverse order. Their example, fittingly, was an airline reservation that pauses for the customer after each seat.

Compensation offsets an effect and leaves it on the record. A refund leaves the charge on the statement; the customer sees both lines. So the rules for agents are practical. Define how to undo each step before the step runs; if nobody can say how to release a hold, the agent shouldn't place it automatically. Make the compensations safe to retry, since they are effects too. Put the steps that can't be undone, such as the email or a wire transfer, after every step that might fail. And let the orchestrating code, which knows what was done, run the unwinding. Reversible Work and Irreversible Effects takes up how much freedom to give actions that can be undone, offset or neither.

Tell the model, and the user, the truth

All of this lives in the harness and the tool layer, not in the prompt. A prompt that says "never retry a payment" is a wish: it depends on the model recognizing the dangerous kind of error, on the instruction still being in its context, and on nothing outweighing it. Same Model, Different Results calls controls like keys and journals walls, because a duplicate charge is just as bad when the model is perfect.

The model still plans the next step, so give it a result it can't misread. Compare "Error: timeout after 5s" with:

status: unknown
The charge for invoice INV-2291 timed out after 5 s. The provider may have processed it.
The harness is checking (reference renewal-8812:3) and will report the outcome.
Do not start another charge for this invoice.

The message lets the model plan well, for instance by telling the customer the payment is being checked. A block in the harness that refuses a new charge on that invoice until the step is resolved makes the last sentence true even if the model misreads it.

Users need the same three states. Done, with evidence: a charge ID, a message ID, a booking reference. Not done, safe to try again. Unknown, being checked, with when the answer will come. Collapse "unknown" into "failed" and the duplicate moves to the human: a customer told "payment failed" presses Pay again.

How much of this you need

None of it is free. A journal adds a durable write around every effect. Keys take storage. Durable execution platforms impose rules on how code is written. Reconciliation needs providers that let you look things up.

So let the effort follow the cost of a duplicate. Reads need only sensible retry limits. A duplicate status ping to a team channel costs less than the machinery to prevent it. Payments, customer messages, orders, bookings and anything regulated almost always justify the full treatment.

The strongest objection is that models keep getting better: give a capable model a lookup tool and a clear instruction to check before retrying, and it will usually do the right thing. Provide both, by all means. But at the moment of a timeout the information the model needs doesn't exist anywhere it can reach. "Usually" is a probability, while a key makes the duplicate impossible inside its window. And the failures that matter most, a crash, a restart, a cleared context, remove the very evidence a careful model would use.

My expectation, and it is a forecast, is that this becomes a buying criterion. Agent builders can offer safe retries only on top of providers that support keys, lookups and notifications, so APIs that offer them will increasingly be the ones agents are pointed at. Once retries are safe by construction, the agent can be given more room everywhere else (Giving an Agent Room to Work).

What to do

Ask one question of any agent product: what does your system do when a payment or email call times out? A good answer names keys, a journal, a lookup and an "unknown" state users can see. A bad one mentions the prompt.

Mint keys in the harness, from the task and step, attach them to every attempt, and know when each provider forgets them.

Name your effects so you can find them later, and look them up before retrying once a key has expired.

Journal every step outside the model's context, including the model's own decisions. On resume, replay what's settled.

For multi-step tasks, define each undo before the step runs, and put what can't be undone last.

Show users three states, never two.

Check yourself

Why does retrying after a timeout risk a duplicate charge?

The charge may complete after the harness stopped waiting. The model sees only "error" and calls again, and the provider treats that as a brand new request.

Why must the harness, not the model, create the idempotency key?

The key has to be identical on every attempt at the same intention, across retries and restarts. A model asked for a key would invent a new one each time.

What does the journal add that a good prompt cannot?

It records each decision and outcome outside the model's context. After a crash or a cleared context, the task re-reads settled steps instead of deciding and doing them again.

type Outcome =
  | { status: "done"; result: unknown }     // it happened; result is the evidence
  | { status: "not_done"; reason: string }  // it certainly did not happen
  | { status: "unknown"; reason: string };  // it may have happened

interface Step {
  id: string;       // `${taskId}:${n}`, stable across attempts and restarts
  tool: string;
  args: unknown;    // frozen when the step is first written down
  key: string;      // idempotency key, derived from id, never from the model
  startedAt: number;
  outcome?: Outcome;
}

async function perform(step: Step, journal: Journal, tools: Tools): Promise<Outcome> {
  const tool = tools[step.tool];
  const prior = step.outcome;
  if (prior && prior.status !== "unknown") return prior;  // settled: replay

  if (prior?.status === "unknown") {                      // ask the world first
    const seen = await tool.lookup?.(step.key);
    if (seen) return journal.settle(step.id, { status: "done", result: seen });
    const keyAlive = tool.keyTtlMs !== undefined
      && Date.now() - step.startedAt < tool.keyTtlMs;
    if (!keyAlive) return prior;                          // no safe retry left
  }

  try {
    const result = await tool.call(step.args, { idempotencyKey: step.key });
    return journal.settle(step.id, { status: "done", result });
  } catch (err) {
    // "Not done" needs proof: the server refused (a 4xx other than 409 or 429),
    // or a first attempt never reached it. Anything else may have happened.
    const provenNotDone = err instanceof Rejected
      || (err instanceof NeverReachedServer && !prior);
    const outcome: Outcome = provenNotDone
      ? { status: "not_done", reason: String(err) }
      : { status: "unknown", reason: String(err) };      // timeout, reset, 5xx, 409
    return journal.settle(step.id, outcome);
  }
}

async function runTask(taskId: string, journal: Journal, model: Model, tools: Tools) {
  const finished = await journal.summary(taskId);         // finishing is a decision too
  if (finished !== undefined) return finished;
  for (let n = 1; ; n++) {
    const id = `${taskId}:${n}`;
    let step = await journal.get(id);
    if (!step) {                                          // a new decision
      const history = await journal.outcomes(taskId);    // facts, not memory
      const next = await model.decideNext(history);
      if (next.kind === "finish") return journal.finish(taskId, next.summary);
      step = await journal.record(                       // write before acting
        { id, tool: next.tool, args: next.args, key: id, startedAt: Date.now() });
    }
    const outcome = await perform(step, journal, tools); // settled steps replay
    if (outcome.status === "unknown") return `needs attention: ${id}`;
  }
}