How to reconcile AI tool calls when a timeout hides the result

AI tool timeout reconciliation starts with an uncomfortable fact: a timeout tells the caller that it stopped waiting. It does not tell you whether the remote action failed. The tool may never have received the request, may still be working, or may have completed successfully while its response disappeared on the way back.

That distinction barely matters for a read-only search. It matters immediately when an agent sends an email, creates an invoice, opens a ticket, publishes a post or changes production infrastructure. Treating every timeout as failure can repeat the side effect. Treating every timeout as success can advance a workflow past work that never happened.

The safe state is neither success nor failure. It is unknown. A production workflow needs to persist that uncertainty, gather evidence and choose the next action according to the tool's contract. This guide presents the design I use for that boundary in AI automation and PHP systems.

A timeout is a local observation, not a remote verdict

Imagine an agent asks a provider to create a support ticket. The provider commits the ticket at 10:00:04. Its response crosses a load balancer, but the connection closes at 10:00:05. The worker's five-second client deadline expires and throws an exception.

From the worker's perspective, the request timed out. From the provider's perspective, it succeeded. Both statements are true.

The failure window exists because request execution and response delivery are separate events. An HTTP status can classify a response only when a response arrived. A socket timeout, connection reset or worker termination may leave four possible realities:

1. the request never left the caller;
2. the provider rejected it before starting;
3. the provider started or completed it, but the response was lost;
4. the provider is still processing it.

A generic catch (Throwable) block cannot distinguish them. Neither can an LLM. The model may propose retrying, but it has no new evidence about the remote system. Recovery belongs to deterministic application code with persisted identifiers, provider-specific rules and bounded authority.

This is the difference between retrying transport and reconciling business state. A retry asks, “Can I send a request again?” Reconciliation asks, “What effect already exists, and what evidence proves it?”

Persist intent before calling the tool

The most important record is created before the network call. If the only copy of the operation ID lives in worker memory, a crash erases the evidence needed to investigate the request.

I store one durable operation row for every side-effecting tool call:

The idempotency key identifies a logical action, not a network attempt. If a worker times out and another worker resumes the operation, both must use the same key and the same semantic arguments. Creating a fresh key on every attempt defeats idempotency by telling the provider that every retry is new work.

Persisting the row and scheduling the call should also be atomic. A transactional outbox can commit workflow state and the durable intent to invoke the tool together. It does not resolve the remote outcome, but it prevents the local system from losing the fact that work should be attempted.

Model unknown as a first-class state

Boolean fields such as succeeded cannot represent the boundary accurately. I prefer a small state machine:

pending -> dispatching -> confirmed_succeeded
                    |--> confirmed_failed
                    |--> unknown -> reconciling
                                      |--> confirmed_succeeded
                                      |--> confirmed_failed
                                      |--> manual_review

unknown means the caller lacks a trustworthy terminal result. It is not a synonym for provider failure. While an operation is unknown, the workflow must not independently issue the same business action under a new identity. Downstream transitions that depend on success must wait.

Transitions should be conditional and monotonic. A late success webhook may race with a reconciliation job. Both should be able to confirm the same operation, but neither should move a terminal success back to unknown. Use a row lock, optimistic version or compare-and-set update around the transition.

This structure fits naturally with an explicit AI agent state machine. The workflow knows which business step is blocked. The operation record knows which external effect is uncertain. Keeping them separate lets multiple tool calls exist within one workflow without compressing every failure into a single run status.

Use a reconciliation ladder, not a blind retry loop

The safest evidence source varies by provider. I use a ladder from strongest and cheapest evidence to weakest and most expensive:

1. Retry the same idempotent request

If the provider documents an idempotency contract, retry with the same key and exactly the same arguments inside its retention window. A correct provider can return the original or semantically equivalent result without repeating the effect.

This is stronger than deduplicating inside one PHP process. The key must survive process restarts, queue redelivery and deployment. Its retention period matters too: once a provider prunes the key, resending it may create new work.

2. Query by provider request or resource ID

If the initial response exposed a request ID before the body failed, save it. Poll the provider's status endpoint. For asynchronous APIs, accepted means the command was registered, not that the business effect completed; keep checking until the provider reaches a documented terminal state.

3. Search by your stable correlation value

When creating a remote object, include a local operation ID in a supported metadata, external-reference or client-reference field. Then search using that exact value. A deterministic identifier is much safer than matching “roughly the same recipient, amount and time.”

4. Consume signed events

A webhook or event stream may confirm completion after the synchronous response was lost. Verify the signature, deduplicate the event and correlate it to the operation. Events can arrive late or out of order, so they supplement the state machine rather than bypass it.

5. Compare observable business state

If the provider offers no correlation lookup, inspect the strongest observable result: a message ID in the mailbox, a ticket with a unique external reference, or a deployment with the expected immutable revision. This is domain-specific reconciliation, and its false-match risk must be documented.

6. Escalate instead of guessing

If the evidence remains ambiguous and repetition could be harmful, stop automation. Show an operator the intended action, timestamps, fingerprint, provider evidence and available choices. Human approval is not a generic error handler; it is the correct control when the system cannot safely recover authority on its own.

A practical Laravel implementation

In Laravel, the queue job should orchestrate an operation record rather than treat the provider call as the operation itself. The following abbreviated service claims a pending operation and records ambiguity explicitly:

final class ExecuteToolOperation
{
    public function handle(string $operationId): void
    {
        $operation = DB::transaction(function () use ($operationId) {
            $operation = ToolOperation::query()
                ->lockForUpdate()
                ->findOrFail($operationId);

            if ($operation->isTerminal()) {
                return null;
            }

            $operation->markDispatching(now());
            $operation->save();

            return $operation;
        });

        if ($operation === null) {
            return;
        }

        try {
            $result = $this->tools->execute(
                tool: $operation->tool,
                arguments: $operation->arguments,
                idempotencyKey: $operation->idempotency_key,
            );

            $this->confirmSuccess($operation->id, $result);
        } catch (AmbiguousTransportFailure $failure) {
            $this->markUnknownAndScheduleReconciliation(
                $operation->id,
                $failure->evidence(),
            );
        } catch (DefinitiveProviderRejection $failure) {
            $this->confirmFailure($operation->id, $failure);
        }
    }
}

The exception types are part of the adapter contract. A validation response received before execution can be definitive. A timeout after bytes were sent is ambiguous. An HTTP 500 may also be ambiguous if the provider can commit a side effect before returning the error. Status code alone is not enough; use the provider's documented semantics.

The reconciliation job should be independent, bounded and safe to run more than once:

$evidence = $adapter->reconcile(
    idempotencyKey: $operation->idempotency_key,
    providerReference: $operation->provider_reference,
    correlationId: $operation->id,
);

match ($evidence->status) {
    'succeeded' => $transitions->confirmSuccess($operation, $evidence),
    'failed'    => $transitions->confirmFailure($operation, $evidence),
    'pending'   => $scheduler->checkLater($operation),
    'unknown'   => $budgets->retryOrEscalate($operation, $evidence),
};

Laravel job attempts are not the business retry counter. Infrastructure may redeliver a job without making another provider request, while one job may perform several safe status checks. Persist separate dispatch and reconciliation budgets, as described in retry budgets for AI automation.

When the tool offers no safe retry contract

Some integrations expose only a side-effecting endpoint and a timeout. They accept no idempotency key, return no searchable client reference and publish no reliable event. The correct response is not to hide the problem behind exponential backoff.

First, reduce the ambiguity surface. Split “prepare” from “commit” when the provider supports drafts. Let the agent create a local or remote draft, then require an explicit, separately tracked send operation. Prefer APIs that accept caller-supplied identifiers. Wrap legacy tools behind an internal adapter that records requests and serializes operations per business object.

Second, classify the consequence. Repeating an analytics write may be acceptable. Repeating a payment, public post, email or infrastructure mutation may not be. Safe policy depends on impact, not on how easy the retry is to code.

Third, restrict the agent's authority. The model can explain the desired outcome, but deterministic policy decides whether a retry is permitted. For high-impact actions with weak provider guarantees, route unknown outcomes to the same approval boundary used before other irreversible actions.

Compensation is not a substitute for reconciliation. Deleting a duplicate ticket or refunding a duplicate charge creates another side effect that can also fail. Use a compensating action only after evidence confirms the unwanted effect.

Test the lost response, not only the returned error

The critical test case is easy to miss: make the fake provider apply the effect, then throw a timeout before returning the result. A naive suite often simulates a timeout by preventing the handler from running at all. That tests an ordinary failure, not an ambiguous one.

I want contract and integration tests for at least these sequences:

• timeout before the provider receives the request;
• timeout after the provider commits the effect;
• provider still processing when reconciliation starts;
• same idempotency key replayed with identical arguments;
• same key accidentally reused with different arguments;
• webhook arriving before and after the reconciliation poll;
• two reconcilers racing to confirm the same operation;
• worker crash after remote success but before the local success update;
• idempotency retention window expiring while the outcome is unknown;
• manual review resolving an operation while a delayed event arrives.

Assert business effects, not just job status. The invariant is that one logical operation produces at most one intended external effect, or reaches a visible state where a responsible person can decide what happens next.

Make ambiguity visible in production

Unknown operations should appear in operations dashboards as their own backlog. A generic failed-job count cannot tell you whether an external effect may already exist.

Useful signals include:

• unknown outcomes by tool, action and provider;
• time from ambiguity to confirmation;
• percentage resolved by idempotent replay, lookup, event or human review;
• operations approaching provider idempotency expiry;
• duplicate effects detected after reconciliation;
• reconciliation attempts and budget exhaustion;
• oldest operation in unknown or reconciling.

Log identifiers and state transitions, but avoid copying secrets, full prompts or sensitive tool arguments into logs. A request fingerprint can prove that attempts used the same canonical payload without exposing the payload itself. This belongs in the event model described in AI agent observability.

Production checklist

1. Create a durable operation ID before every side-effecting call.
2. Persist one idempotency key per logical action, not per attempt.
3. Record the canonical request fingerprint and provider references.
4. Classify transport ambiguity separately from confirmed rejection.
5. Block dependent workflow transitions while the outcome is unknown.
6. Reconcile through replay, lookup, correlation metadata and events.
7. Make terminal transitions concurrency-safe and idempotent.
8. Bound dispatch attempts, reconciliation time and total cost.
9. Escalate irreversible ambiguity instead of generating a new identity.
10. Test success-with-lost-response as a normal failure mode.

An agent is not reliable because it retries until an exception disappears. It is reliable when it can preserve uncertainty without inventing an answer, recover evidence after its worker is gone, and avoid spending the same authority twice.

Sources checked October 23, 2026: Amazon Builders' Library on making retries safe with idempotent APIs, Stripe documentation on network errors, indeterminate outcomes and reconciliation, and Microsoft Azure Architecture Center on the Retry pattern. Provider semantics differ, so verify the current contract of every tool you integrate. This article reflects practical experience with PHP backends, queues, automation and AI agents without identifying an employer or client. It was prepared with AI assistance and editorial review. Cover: computer server photograph from Pixabay.

If your AI workflow can trigger real side effects, I can help review its operation state, retry policy and recovery paths. See my AI engineering consulting work or contact me.