Laravel and LLMs: queues, streaming responses and failure handling

Laravel and LLMs work well together when three responsibilities stay separate: queues own slow work, streams deliver progress, and durable application state owns the result. A browser disconnect should not silently erase a generation. A worker retry should not quietly produce a second business action. Streaming text is neither proof of completion nor permission to act.

Imagine a support agent asking an application to draft a reply. The first words appear, the answer grows, and then the connection drops. There is now half a paragraph on screen. Did generation stop? Is the worker still running? Will pressing the button again create another request?

That uncertainty is the design problem worth solving before choosing a streaming library.

The support-draft workflow below is an illustrative design walkthrough, not a report of a particular client incident or a benchmark. The Laravel API references use the versioned 12.x documentation, checked on September 8, 2026. Check your installed framework and provider SDK before adapting the details.

Choose who owns the generation

A direct streamed HTTP response can be a sensible choice for a short, disposable interaction. The controller calls the model and forwards the output. There are fewer moving parts, and cancellation may naturally follow the lifetime of the request, depending on the runtime and provider integration.

The limitation is ownership. Under a conventional PHP-FPM deployment, that long request occupies a worker. If the connection disappears, keeping the work alive and recovering its output requires additional design. A stream helper does not turn a request into a durable background task.

For a draft that users expect to find after refreshing the page, I would choose a background operation:

1. The browser submits an authenticated request to create a draft operation.
2. The application records it and schedules a queue job.
3. A worker calls the model and saves progress and the final result.
4. The browser observes the operation through polling, SSE or broadcasting.

The important distinction is not “queues are good, controllers are bad.” It is whether the result must outlive the connection. A throwaway text experiment and a document-processing workflow have different requirements.

Moving generation into a queue also does not make browser connections free. An SSE endpoint served through PHP-FPM still consumes capacity while it stays open. Polling can be the simpler first implementation; higher connection volumes may justify a dedicated delivery layer. My earlier article on Laravel queues in AI automation covers the broader queue architecture.

Queue an operation, not a browser session

The queue payload should identify a durable operation, rather than carry an entire conversation and depend on request-local state. Store the input snapshot or a versioned reference under the application's normal access controls. A retry should not accidentally read a support ticket that has changed since the original request.

A useful operation record includes an ID, tenant and requesting user, input version, status, active attempt ID, timestamps, result reference and a safe error category. An attempt is one execution of generation; an operation is the user's request. Keeping those identities separate makes retries understandable.

For this example, the lifecycle could be:

queued → running → validating → completed
                  ↘ retry_wait → running
                  ↘ needs_review
                  ↘ failed
queued / running / retry_wait → cancelled

These are application states, not Laravel's built-in job states. Define allowed transitions and enforce them with transactions or conditional updates. A late worker must not overwrite a cancellation or the result of a newer attempt.

Laravel's after-commit dispatch prevents a worker from reading an operation before its creating transaction commits. For example, an application job might be dispatched like this:

GenerateSupportDraft::dispatch($operationId)
    ->onQueue('llm')
    ->afterCommit();

This is a dispatch fragment, not a complete implementation. The job class, authorization, operation persistence and a real asynchronous queue connection must exist. With the synchronous driver, dispatch does not provide a separate background worker.

After-commit dispatch also does not make the database commit and a Redis enqueue one atomic action. A crash between them can leave an operation queued in your database but absent from the broker. Where that gap matters, use a transactional outbox or a reconciler that safely redispatches stranded operations.

Set timeouts as a budget

LLM requests can have a slow first response, long pauses between chunks, or long total duration. Treat connection timeout, read inactivity and total execution time as different constraints. Which settings are available depends on the HTTP client and SDK.

For one bounded model call, an illustrative budget might allow 60 seconds for the provider request, 90 seconds for the job, and 120 seconds before the queue reservation expires. These are example values, not production defaults or latency claims. Leave room for validation and saving the result.

The key relationship is that the effective worker/job timeout must be shorter than the queue's retry_after, with a margin. Otherwise, another worker may receive the job while the first one is still processing it. Laravel documents this explicitly under job expirations and timeouts. SQS uses its visibility timeout instead of Laravel's retry_after setting.

Account for nested retries, too. Three provider attempts of 60 seconds cannot fit inside a 90-second job. Either let the queue own the retry policy or include every SDK attempt and delay in the budget.

Laravel's worker timeout mechanism requires PCNTL support; blocking network calls still need client-level timeouts. Verify the actual worker environment and process manager. Increasing a browser or proxy timeout alone does not fix a job whose reservation expires too early.

Streaming responses are a delivery channel

Separate starting work from watching it

For a queued design, use an authenticated POST to create the operation and return its ID with a 202 Accepted response. A separate read endpoint supplies current state. An optional SSE endpoint supplies updates for that same operation.

Do not start a new generation when the browser opens the event stream. Native EventSource reconnects automatically, so a reconnecting subscription must remain a read operation.

Native EventSource does not accept an arbitrary POST body or custom authorization headers. Same-origin session authentication can fit this design; a fetch-based streaming client may fit a bearer-token API better. Authorize every status and stream request against the operation's tenant and user. An unguessable ID is not authorization.

Laravel provides event-stream responses, but the helper does not automatically connect a queue worker to a browser. You still need an event store, shared progress state or a messaging layer between those processes.

Reconnect to saved progress, not a new answer

An application-defined SSE event could look like this:

id: 42
event: draft.delta
data: {"operation_id":"op_123","attempt_id":"try_2","text":"Hello"}

The ID orders persisted events for an operation. The attempt ID prevents the interface from combining text from two different generations. These names are an example protocol, not Laravel defaults.

MDN documents SSE event IDs and reconnection. The browser can send its last event ID when reconnecting, but the server must retain and replay missed events. The protocol does not provide storage. Plain ephemeral pub/sub cannot recover messages emitted while a subscriber was offline.

If the requested cursor has expired, return a snapshot/reset instruction and the latest durable state. For a completed operation, the saved result is enough. On a fresh page load, fetch a snapshot with its cursor before following later events. Deduplicate replayed events and avoid a gap between reading history and subscribing to new updates.

Test through the real proxy path. Buffering can turn incremental output into one large response; idle timeouts can cut off a quiet stream. Heartbeats help only if they reach the client. Bound event retention and coalesce tiny text deltas so the progress log does not become an expensive second copy of every conversation.

Treat partial output as a preview

A paragraph appearing on screen is a preview, not a committed result. If generation fails halfway through, label the text incomplete. Do not leave it looking like a finished answer merely because it ends with punctuation.

Structured output makes this distinction more important. An incomplete JSON object is not valid input for a business action. Even a syntactically complete object may lack required fields, contain unsupported claims or violate application rules.

Wait for the provider's documented completion signal, assemble the result, validate it and persist the accepted version before publishing the application's completion event. A transport closing normally is not sufficient evidence that generation completed successfully.

For a support draft, validation might check the expected structure, ticket association and references. Sending the reply should be a separate authorized action. A model-generated tool request must likewise pass authorization and argument validation; partial tool arguments must never trigger execution.

Render model text as untrusted content. Escape plain text, or sanitize Markdown-derived HTML with a controlled policy. Streaming should not become a shortcut around the rendering protections used elsewhere in the application. The same boundary belongs in tests for AI-generated output.

Failure handling needs separate recovery paths

“Try again” can mean reconnecting a browser, rerunning inference, or repeating a downstream action. Those operations have different costs and risks. Give them different recovery paths.

Failure	What remains useful	Recovery
Browser connection drops	Running operation and retained events	Reconnect or read status; do not regenerate
Provider rate limit	Input snapshot and operation ID	Schedule bounded backoff; respect provider guidance
Generation stops halfway	Incomplete preview and attempt history	Mark interrupted; create a distinct attempt if retryable
Validation rejects output	Candidate result and validation reason	Review or bounded correction, not endless replay
Result saved, notification lost	Completed durable result	Recover delivery from saved state
External action has an unknown outcome	Action ID and request evidence	Reconcile before repeating the action

Classify provider failures rather than retrying every exception. A temporary capacity problem differs from invalid credentials or unsupported input. Apply bounded backoff with jitter, a deadline and an attempt budget. Rate-limit middleware releases also consume queue attempts, so configure limits with that behavior in mind.

Coordinate capacity across workers using the same provider account. Adding workers during a provider rate limit can increase rejected requests without increasing completed work. Separate interactive drafts from bulk processing where their latency requirements differ.

A retried generation can produce different text. Never append the second attempt to the first attempt's preview. Replace or explicitly version the preview, while keeping the same operation identity.

Likewise, duplicate job delivery must not duplicate accepted results or downstream actions. Use conditional state transitions, database uniqueness and stable action-level idempotency keys where supported. Queue uniqueness locks are useful coordination, not a substitute for the idempotency contract. Exhausted work needs an owned recovery workflow, not just a failed-job counter.

Make cancellation and completion explicit

Closing a tab should not implicitly cancel a durable task. Provide a cancellation endpoint that records intent, checks authorization and lets the worker observe it. Where supported, abort the provider request as well, but do not promise that cancellation reverses work or removes charges already incurred.

Cancellation races with completion. Define which transition wins through a conditional write against the operation's current state and active attempt. If cancellation has already been accepted, a late result must not become the published answer.

Completion has a similar ordering requirement: save the validated result first, then notify subscribers. If notification fails, the status endpoint still returns the answer. If saving fails, the interface must not announce success.

A killed worker may never execute application cleanup. Track stale running operations with a lease or heartbeat and reconcile them after a sensible deadline. Recovery should consult durable state, not assume a failure callback always ran.

Test the boundaries before adding workers

Happy-path token streaming proves very little about recovery. Before expanding capacity, I would test the exact boundaries the design relies on:

• Disconnect after a few events. Reconnect and verify that no second model call starts and no text is duplicated.
• Drop the final notification. The status endpoint must still return the saved result.
• Kill a worker during generation. Reconciliation must identify stale work, and the next attempt must not merge previews.
• Deliver a job twice. Verify one accepted result and one downstream action, not merely one log entry.
• Race cancellation with completion. The stored state and user-visible result must agree.
• Expire the event cursor. The browser must recover through a snapshot rather than wait forever.
• Request another tenant's operation. Both status and stream endpoints must deny access.
• Commit the operation while enqueueing fails. The outbox or reconciler must restore progress.

Use a controllable provider fake for delayed chunks, malformed output and disconnects. Queue fakes can prove dispatch intent, but they cannot prove worker timeouts, broker redelivery or reverse-proxy streaming. Exercise those with the actual queue driver and deployment path in an integration environment.

Measure queue wait, time to first visible output, total generation time, validation failures, retries and abandoned operations separately. A fast first token can hide a very slow queue or a high final failure rate. Correlate events by operation and attempt without copying raw prompts into every log. That is where event-based AI observability becomes useful.

Questions worth settling before implementation

Does every LLM call need a queue?

No. Direct streaming can suit short, disposable interactions when request capacity and disconnect behavior are acceptable. Choose a queue when the work must survive the browser, needs independent scheduling, or shares constrained provider capacity with other workloads.

Can SSE resume the provider's generation?

Not by itself. SSE can reconnect the browser to your application. Replaying your saved events is different from resuming a model request. Provider-side continuation or retrieval is a separate capability that must be verified for the API you use.

Should every token be persisted?

No. Persist the final result and enough progress to meet the recovery promise. Chunked events or periodic snapshots may be sufficient. Decide retention and replay granularity deliberately; a durable operation does not require permanent storage of every token boundary.

The useful abstraction is the operation

Laravel gives you queue workers, response helpers and failure hooks. The application still has to explain what happens between “accepted” and “finished.”

Start with one operation ID, explicit attempts, a saved result and an honest status endpoint. Add streaming when it improves the experience. Then break the connection, interrupt the worker and lose the notification. If the user can still understand and recover their work, the design is doing its job.

For more background on my engineering focus, see about me. The principle here is simple: a stream should show progress, not own the only copy of it.

Technical references: Laravel 12.x queues and HTTP responses; MDN's SSE guide. Sources checked September 8, 2026. Cover: network router photograph from Pixabay.