Token budgets are not enough: cost controls for production AI agents

Cost controls for production AI agents cannot stop at a token counter. A token limit constrains one model request. A production agent is a workflow: it retrieves context, calls models, invokes tools, retries failures, waits on queues and may delegate work to other agents.

I learned to treat cost as execution state after seeing a familiar pattern in automation systems. Every individual job looked inexpensive. The surprise appeared only after retries, growing context and parallel branches were added together. Nothing was obviously broken, yet the cost of one business outcome had become unpredictable.

The fix is not another dashboard. The fix is a budget that belongs to the run, travels through every queue message and is enforced before the next expensive action begins. This article explains the architecture I use to build that control into PHP and Laravel workflows.

Why a token budget is not a cost budget

Tokens are useful accounting units, but they are not the same as money and they are not the same as value. Input, output, cached and reasoning tokens can have different prices. Different models price those units differently. Tools may add search, browser, storage or third-party API charges that a model token counter never sees.

More importantly, a per-request limit cannot see the workflow around the request. Imagine a research agent with a 20,000-token ceiling. It uses 8,000 tokens, hands work to a summarizer and schedules a verification job. If every component receives its own fresh 20,000-token allowance, the nominal cap has quietly become 60,000 tokens.

The same problem appears with retries. A provider timeout may produce another call. A queue redelivery may repeat a step. A tool returns weak data, so the planner searches again. Each call stays below its local limit while the run exceeds what the result is worth.

This is why I distinguish four controls:

Google Cloud's token-management documentation makes the same operational distinction: request-count limits are insufficient when request sizes vary, so usage must be attributed and aggregated into shared token groups. Microsoft describes resource governance as bounding tokens, steps, spawned agents, tool calls, elapsed time, retries and total spend. The common idea is simple: constrain the execution graph, not only the model response.

Model the full cost of an agent run

Before enforcing a budget, define what it includes. I use a run as the unit of accountability because it maps to something the product can name: resolve a ticket, review a document, enrich a lead or produce an accepted pull request.

A practical cost equation is:

run cost = model calls
 tool and search calls
 retrieval and embedding work
 sandbox or browser compute
 retry amplification
 human review, when comparing full economics

The online ledger normally records machine costs. Human time belongs in later unit-economics reporting, not in the hot-path circuit breaker. Mixing the two makes enforcement slow and subjective.

Every charge event should carry enough dimensions to explain itself:

• run_id, workflow name and tenant or product;
• step, attempt and parent branch;
• provider, model and pricing version;
• input, cached, output and reasoning units where available;
• tool name and billable units;
• estimated cost before admission and realized cost afterward;
• the policy decision that allowed, changed or rejected the action.

Keep the pricing version. Recalculating an old run with today's model prices rewrites history. Store the usage facts and the rate card used for the original estimate, then reconcile with the provider bill separately.

Use five layers of cost control

No single threshold covers every failure mode. AWS recommends layered budgets per cycle, task and day, combined with anomaly detection and cost-quality analysis. I turn that idea into five enforcement layers.

1. Request bounds

Cap input size, output size and the most expensive model class a step may use. Reject accidental full-database context before it reaches the provider. These checks are cheap and deterministic.

2. Execution bounds

Limit steps, wall-clock duration, retries, tool calls, recursion depth and parallel fan-out. A dollar cap alone reacts late when the price estimate is stale. Structural limits stop obvious loops early.

A retry budget is especially important. Ten cheap failures can cost more than one deliberate fallback. Retries should spend from both a retry allowance and the shared monetary budget.

3. Run budget

All branches reserve against one authoritative ledger. The budget follows the run through queues, sub-agents and provider boundaries. It never resets because a new process handles the next step.

4. Tenant and fleet limits

A well-behaved run can still be part of a traffic spike. Add per-tenant hourly limits, per-workflow daily limits and a fleet emergency ceiling. Use concurrency limits and queue backpressure alongside spend caps so autoscaling does not simply multiply an expensive mistake.

5. Outcome economics

Track cost per completed and accepted outcome. This layer decides whether the workflow deserves its budget. It is an offline product decision, not a model deciding whether it may ignore a financial control.

Build one shared ledger for the entire run

The hardest bug in cost control is usually not arithmetic. It is concurrency.

Suppose two sub-agents read a remaining budget of $0.40. Each estimates its next call at $0.30, so both proceed. The run spends $0.60 even though every local check passed. A counter stored inside each worker makes the same mistake at a larger scale.

The solution is an admission reservation:

1. estimate the worst reasonable cost of the next action;
2. atomically reserve that amount against the run;
3. execute only if the reservation succeeds;
4. replace the reservation with realized cost;
5. release unused value or expire abandoned reservations.

The invariant is:

realized_spend + active_reservations <= hard_budget

A reservation must have an idempotency key. Queue redelivery should find the existing reservation, not create a second one. Completion must also be idempotent because a worker can record the provider result and then lose its acknowledgement.

This ledger belongs beside the workflow state, not inside an analytics pipeline. Analytics can be eventually consistent. Admission cannot. If a strict budget is a contractual or safety boundary, enforce it with a transactional row, atomic Redis script or another strongly consistent primitive.

The failure policy must be explicit. For a low-risk internal summarizer, a temporarily unavailable ledger may fail open under a small emergency allowance. For an autonomous workflow that can fan out or buy external services, fail closed and move the run to an operator-visible waiting state.

A Laravel implementation with reservations

In Laravel, I keep the policy outside the provider SDK. Model and tool clients call one budget service before dispatch. That prevents each integration from inventing different rules.

final class ReserveAgentCost
{
    public function handle(
        string $runId,
        string $operationKey,
        Money $estimatedMaximum,
    ): CostReservation {
        return DB::transaction(function () use (
            $runId,
            $operationKey,
            $estimatedMaximum,
        ) {
            $existing = CostReservation::query()
                ->where('operation_key', $operationKey)
                ->first();

            if ($existing) {
                return $existing;
            }

            $budget = AgentRunBudget::query()
                ->lockForUpdate()
                ->findOrFail($runId);

            if (! $budget->canReserve($estimatedMaximum)) {
                throw new AgentBudgetExceeded($runId);
            }

            $budget->reserve($estimatedMaximum);
            $budget->save();

            return CostReservation::create([
                'run_id' => $runId,
                'operation_key' => $operationKey,
                'reserved_micros' => $estimatedMaximum->micros(),
                'status' => 'active',
                'expires_at' => now()->addMinutes(10),
            ]);
        });
    }
}

I store money as integer micros in one settlement currency. Floating-point dollars eventually create reconciliation noise. The estimate uses the selected model, expected input and maximum allowed output. Tool reservations use their own unit model.

After the call, a second transaction settles actual cost and releases the difference. If the result is ambiguous, do not release the reservation immediately and retry. First reconcile the provider operation, just as you would reconcile an AI tool call hidden by a timeout.

Dispatch charge events through a transactional outbox. The workflow transition and its cost event should commit together. Otherwise dashboards may report a cheap run simply because telemetry was lost between the database commit and the event broker.

Degrade before you halt

A hard stop is necessary, but it should be the last action. AWS describes graduated throttling as a way to preserve useful autonomy under cost pressure. I implement three thresholds:

Degradation must be defined per workflow. Blindly switching every step to a cheaper model can lower quality, trigger retries and increase total cost. Model routing is useful when tested against representative cases, not when it is an emergency guess.

The agent should never negotiate its own hard limit through prompt text. The model can propose a plan, shorten an answer or request more budget, but deterministic application code decides. If an extension is justified, record who approved it, how much was added and why.

When a run stops, save a checkpoint with the reason, remaining work and safe resume command. The cost circuit breaker should integrate with agent checkpoints, not turn an expensive run into discarded work.

The metric that matters is cost per useful outcome

Optimizing cost per token rewards the wrong behavior. A cheap model that completes 60% of tasks and causes repeated review can be more expensive than a stronger model with a higher per-call price.

Use a small scorecard:

• cost per attempted run;
• cost per technically completed run;
• cost per accepted outcome;
• human review minutes per accepted outcome;
• p50, p95 and maximum run cost;
• budget-exhaustion and manual-escalation rates.

Compare those metrics by workflow version, model route and tenant segment. Averages hide runaway tails. The p95 and maximum tell you whether controls are actually bounding risk.

Quality evaluation stays beside cost evaluation. A hard budget knows that a run spent $0.80; it does not know whether the generated patch passes tests or the answer is correct. My production agent evaluation process treats correctness, safety, latency and cost as separate signals. The release decision uses all of them.

Roll out cost controls without breaking active agents

Start in shadow mode. Record what each admission decision would have done without changing execution. This establishes realistic distributions and reveals missing cost sources.

1. Instrument: attach a run ID and pricing version to every model and tool call.
2. Baseline: measure cost and completion by workflow version for a full business cycle.
3. Shadow: evaluate limits without enforcing them.
4. Warn: alert and mark runs that cross soft thresholds.
5. Steer: enable tested context trimming, routing and fan-out reduction.
6. Enforce: activate hard admission limits for a canary cohort.
7. Review: compare cost, quality and escalation before expanding.

Put budget policies in versioned configuration. A long-running agent should keep the policy contract it started with unless a clearly defined emergency ceiling overrides it. Silent policy changes make historical comparisons unreliable and can strand active runs.

Add regression gates to CI. Run a representative evaluation set and fail the build when p95 cost rises beyond an agreed tolerance without a corresponding quality gain. This catches a prompt that doubles context or a new tool loop before production traffic pays for it.

Finally, connect alerts to actions. An alert that says “spend is high” after the run finishes is accounting. A control that disables fan-out, pauses the run and leaves a resumable checkpoint is operations.

Production checklist

• Use a stable run ID across queues, processes, agents and providers.
• Record model, tool, pricing version, step and attempt for every charge.
• Reserve estimated maximum cost atomically before expensive work.
• Settle realized cost idempotently and expire abandoned reservations safely.
• Bound tokens, steps, retries, tools, elapsed time, fan-out and concurrency.
• Define soft thresholds, graceful degradation and a hard stop.
• Persist checkpoints and an operator-visible reason when a run stops.
• Measure p95 cost and cost per accepted outcome, not only total tokens.
• Version budget policies and test cost-quality regressions before release.
• Reconcile the internal ledger with provider billing.

Production AI agents need the same discipline as any other distributed system that can consume money: ownership, limits, atomic admission, observable state and recovery. Token budgets are one sensor inside that system. The real control is a shared, enforceable contract over the entire run.

Sources and further reading

• Microsoft: real-time, run-scoped cost control for AI agents
• AWS Agentic AI Lens: cost governance and continuous optimization
• Google Cloud: token management policies overview
• Microsoft Learn: resource governance and abuse prevention