Why AI Agents Need Evidence Before They Retry

How to prevent a lost response from becoming a duplicate action

A support agent authorized to generate one refund sends the request but loses the connection. The agent receives no confirmation of success and attempts to send the refund again. If the original request was successful, then the agent’s repeat action may result in a second refund.

This is purely hypothetical. The example represents a problem that cannot be solved by model accuracy alone. What actions should an agent take if it cannot determine whether its previous action changed the state of the environment?

Unite.AI has explored agent reliability beyond models. Recovery issues deserve their own rules in those discussions. An agent should confirm what occurred, or have a valid guarantee against duplicate effects, before attempting a consequential action for a second time.

An Unreturned Response Does Not Establish Failure

An unreturned response does not establish that an action has failed. It does not indicate whether an error occurred before execution, whether the action completed without acknowledgement, or whether the process is ongoing.

This is a common issue in distributed systems, as described by Amazon. Amazon explains why retry mechanisms need specific safeguards against repeated side effects. Agents add a new decision-making entity to your system’s recovery path. They can make additional decisions about how to recover from errors.

These decisions can lead to changes in the manner in which business operates. For example, retrying a refund with a different dollar amount is not just another iteration of the same request. Likewise, choosing a second payment service when the first one stopped responding is not simply another iteration of the same request.

Teams must differentiate between granting permission to execute an action and granting permission to recover from an unknown outcome. The original approval does not imply that executing the action again is safe.

Recent Research Measures the Risk

A September 2026 preprint, “Where Does Exactly-Once Live?”, introduced the LIMBO benchmark and tested 25,930 episodes on nine models. Its simulated services were tested with lost acknowledgements, delayed processing, and duplicate deliveries.

Out of 1,279 episodes where a duplicate effect was generated in the authors’ main test, agents reported successful completion in 90%. An agent may therefore report success even when incorrect processing has occurred.

These results are based on controlled benchmarks and do not estimate production failure rates. The services were simulated, and several analyses were exploratory. Teams should evaluate the results of agents’ actions separately from their reports of successful execution.

The paper also addresses why checking the current state can be insufficient. A request still being processed may take effect after a check. Even a blank status response does not mean that the request will not eventually take effect.

Repeated Attempts Refer to the Same Intent

A receiving service has a method of recognizing repeated attempts at one operation with an idempotency key. If properly supported, this method ensures that repeated attempts do not create additional effects. The idempotency key must remain stable during all retries.

Stripe’s API documentation provides examples of both the mechanisms and limitations of idempotency keys. For API v1, keys can be deleted after at least 24 hours. Using a pruned key again will cause a new request. Stripe also verifies that the parameters provided by the requester match those in the initial request.

The overall practical application of idempotence is that it exists within your integration layer, not just within a prompt. Before you dispatch an operation, record the approved operation. Store its identifier, the exact parameters, the provider reference if applicable, and the status of the operation.

For a refund, store the connection of the approval to the original transaction and any returned provider refund identifiers. Searching for any refund of the same amount is less convincing proof because two legitimate requests could have the same amount. There could also be a delay between a refund request and its appearance in a search result. The original request could still be running.

When an agent or worker restarts, the execution layer should retain that identity. It should reject a modified request submitted as the same operation. Such a change represents a new action that requires a new approval decision under the workflow rules.

Tool Descriptions Are Not Enough

In addition to tool descriptions, verify the receiving service’s contract for idempotency support. Test its implementation to confirm what guarantees are actually being met.

Treat Uncertainty as an Operating State

Figure 1. A proposed recovery path for consequential actions. Every retry also requires current authorization.

Confirmed completion means that the workflow will document the result and continue without repeating the action. Confirmed non-execution allows another attempt only if no original request remains pending and the authorization remains in effect.

There is much more to consider about an unresolved outcome. If there is a valid provider-supported guarantee against duplicate effects, you may allow the original request to be retried with the same key. Otherwise, suspend writing and reconcile through a status query, provider support, or an authorized operator. An inconclusive check does not provide sufficient reason to authorize another attempt.

Pausing can also be justified when the provider documents processing and visibility bounds. Without those bounds, elapsed time alone will not confirm that the action did not execute. You have to define who owns cases that do not get resolved and when those owners must review them.

Recovering from Failure May Require a Different Business Process

If a refund is successful but the subsequent customer notification fails, repeating the entire workflow could repeat the refund. Recovery should focus on completing the unfinished step while preserving evidence of completed work.

If corrective actions are needed for previously completed actions, Microsoft provides guidance on compensating transactions. Each application determines the correct way to compensate for errors. Compensation may not necessarily put things back to where they started, and compensation itself may fail.

Additionally, canceling a transaction may result in a charge. Correcting a message cannot ensure that someone did not see an earlier version. Therefore, any recovery activity must explicitly limit what the agent can undo, change, or commit to. Decisions outside these limitations must be routed to an individual with the necessary authority.

Test the Second Attempt Before Deploying

Choose one consequential tool and perform deliberate recovery testing in a test environment. After successful execution, drop the response to the original request. Delay the first request’s execution until after a status check. During recovery, restart the worker. Wait for key retention to expire before attempting again. Make one parameter change during repeated attempts.

Evaluate the resulting business records rather than simply relying on tool responses. Record duplicate effects, unresolved operations, missed work, recovery time, and additional manual interventions. Calculate cost per correctly completed task using total workflow costs, including transaction reversals and customer remedies.

In addition, test a genuine second request similar to the original. Ensure that your system avoids creating duplicates without preventing new work. Assess duplicate avoidance and eventual task completion separately. Otherwise, an agent that terminates every task could appear safe while delivering little or no value.

Your evaluation question should be clearly stated. Was the system able to explain:

• Which operation was authorized?
• What evidence existed to demonstrate its outcome?
• Why was it permissible to make another attempt?

An agent that pauses due to unresolved uncertainty may appear less effective during demonstrations. However, in production environments, pausing can preserve the customer’s original intent. Useful persistence implies that the system completes authorized tasks without silently performing them in duplicate.