A Laravel OpenAI integration looks like one HTTP call, and in a demo it is. In production that call takes several seconds, sometimes fails with a 429, and costs money every time it runs. If it sits inside a controller, your users wait on it, your PHP workers pile up behind it, and nobody can tell you next month which feature ran up the bill.
This guide covers the three things I set up before any LLM feature ships: queued calls for work the user doesn’t need to watch, streaming for work they do, and a usage table that records tokens and cost per feature. It uses Laravel’s own HTTP client, so there’s no SDK to learn, and the same code works with any provider that speaks the OpenAI chat completions format.
What this post covers
- Tested versions
- A small LLM client on Laravel’s HTTP client
- Queued calls for background work
- Streaming responses to the browser
- Tracking tokens and cost per feature
- Testing it without spending a cent
- FAQ
Tested versions
| Tool | Version |
|---|---|
| Laravel Framework | 13.40.0 (fresh laravel/laravel skeleton) |
| PHP | 8.4.27 CLI |
| Composer | 2.8.8 |
| Database | SQLite (the skeleton default) |
| Tests | PHPUnit through php artisan test, 6 tests passing |
Every code block below comes from that app. The tests fake the API, so nothing here was checked against a live account’s rate limits or prices. Treat model names and price numbers as placeholders and copy the real ones from your provider.
A small LLM client on Laravel’s HTTP client
Start with config. The key lives in .env, and so do the prices, because they change and you don’t want a deploy every time they do.
'openai' => [
'key' => env('OPENAI_API_KEY'),
'url' => env('OPENAI_URL', 'https://api.openai.com/v1'),
'model' => env('OPENAI_MODEL', 'gpt-4o-mini'),
'timeout' => (int) env('OPENAI_TIMEOUT', 60),
// Prices per 1M tokens, copied from your provider's pricing page.
'price_in' => (float) env('OPENAI_PRICE_INPUT', 0),
'price_out' => (float) env('OPENAI_PRICE_OUTPUT', 0),
],
Then one class that every feature goes through. Keeping it to one class matters more than it looks: it’s the only place that knows the URL, the timeout and the response shape, and it’s where usage gets recorded, so no feature can skip it.
namespace App\Services\Llm;
use App\Models\LlmUsage;
use Illuminate\Http\Client\PendingRequest;
use Illuminate\Support\Facades\Http;
class LlmClient
{
private function http(): PendingRequest
{
return Http::baseUrl(config('services.openai.url'))
->withToken(config('services.openai.key'))
->timeout(config('services.openai.timeout'))
->acceptJson();
}
/** One request, one full answer. Use this inside queued jobs. */
public function chat(array $messages, string $feature, ?int $maxTokens = 500): string
{
$response = $this->http()
->post('/chat/completions', [
'model' => config('services.openai.model'),
'messages' => $messages,
'max_tokens' => $maxTokens,
])
->throw();
$this->recordUsage($feature, $response->json('model'), $response->json('usage', []));
return (string) $response->json('choices.0.message.content');
}
Two details. ->throw() turns any 4xx or 5xx into a RequestException, which the queued job below uses to decide whether to retry. And the timeout is set on purpose: the default can leave a worker hanging on a slow model far longer than you’d want. If you haven’t set retries and timeouts on outbound calls before, building resilient HTTP clients in Laravel goes deeper on that.
Queued calls for background work
Most LLM work doesn’t need to happen while the user waits. Summarizing a ticket, tagging a lead, drafting a reply someone will review later: all of that belongs in a job. The request saves the record and returns in milliseconds, and a worker makes the slow call.
namespace App\Jobs;
use App\Models\TicketSummary;
use App\Services\Llm\LlmClient;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Http\Client\RequestException;
use Illuminate\Queue\Middleware\RateLimited;
class SummarizeTicket implements ShouldQueue
{
use Queueable;
public int $maxExceptions = 3;
public int $timeout = 90;
public function __construct(public TicketSummary $ticket) {}
/** Keep trying for 10 minutes; rate-limit releases don't count as failures. */
public function retryUntil(): \DateTime
{
return now()->addMinutes(10);
}
/** Wait longer between each failed attempt: 10s, 30s, then 90s. */
public function backoff(): array
{
return [10, 30, 90];
}
public function middleware(): array
{
return [new RateLimited('llm')];
}
public function handle(LlmClient $llm): void
{
if ($this->ticket->summary !== null) {
return; // Already done on an earlier attempt.
}
try {
$summary = $llm->chat([
['role' => 'system', 'content' => 'Summarize the support ticket in two sentences.'],
['role' => 'user', 'content' => $this->ticket->body],
], feature: 'ticket-summary', maxTokens: 120);
} catch (RequestException $e) {
// 400, 401 and 404 will fail the same way every time. Don't retry them.
if (in_array($e->response->status(), [400, 401, 403, 404], true)) {
$this->fail($e);
return;
}
throw $e; // 429 and 5xx: let the queue retry with backoff.
}
$this->ticket->update(['summary' => $summary]);
}
}
A few choices here are deliberate:
- Not every error deserves a retry. A 429 or a 500 usually clears up, so the job rethrows and the queue tries again after the backoff. A 400 or 401 means the request or the key is wrong, and retrying just burns attempts, so the job fails straight away.
- Rate limiting at the job level. The
RateLimitedmiddleware uses a limiter defined once inAppServiceProviderwithRateLimiter::for('llm', fn ($job) => Limit::perMinute(60)). Set the number to whatever your provider allows. The Laravel queue docs point out that a rate-limited release counts as an attempt, which is why the job usesretryUntil()and$maxExceptionsinstead of a fixed$tries. Otherwise a busy minute could use up every try without a single real failure. - The early return makes retries safe. If an earlier attempt already saved a summary, a retry doesn’t pay for a second one.
max_tokensis set per feature. A two-sentence summary doesn’t need 500 tokens of room. Capping output is the simplest cost control there is.
Dispatching is one line from wherever the ticket gets created:
$ticket = TicketSummary::create(['body' => $request->input('body')]);
SummarizeTicket::dispatch($ticket);
Give these jobs their own queue if they run alongside time-sensitive work like password-reset emails, so a burst of AI jobs can’t delay everything else. Laravel queue workers in production covers Supervisor, separate queues and scaling workers.
Streaming responses to the browser
Some answers the user is waiting for: a chat reply, an explanation they asked for. Queuing those doesn’t help, because they’d just poll for the result. Streaming does. The first words show up as soon as the model produces them, and the wait feels far shorter even though the total time is the same.
Pass 'stream' => true in the request body so the API sends server-sent events, and ['stream' => true] as a Guzzle option so Laravel doesn’t wait for the whole body before handing it to you. Then read the body in pieces and pull the text out of each event:
/** Streams the answer, yielding text chunks as they arrive. */
public function stream(array $messages, string $feature, ?int $maxTokens = 500): \Generator
{
$response = $this->http()
->withOptions(['stream' => true])
->post('/chat/completions', [
'model' => config('services.openai.model'),
'messages' => $messages,
'max_tokens' => $maxTokens,
'stream' => true,
'stream_options' => ['include_usage' => true],
])
->throw();
$body = $response->toPsrResponse()->getBody();
$buffer = '';
while (! $body->eof()) {
$buffer .= $body->read(1024);
// Server-sent events are separated by a blank line.
while (($pos = strpos($buffer, "\n\n")) !== false) {
$event = substr($buffer, 0, $pos);
$buffer = substr($buffer, $pos + 2);
foreach (explode("\n", $event) as $line) {
if (! str_starts_with($line, 'data: ')) {
continue;
}
$data = substr($line, 6);
if ($data === '[DONE]') {
return;
}
$chunk = json_decode($data, true);
if (! empty($chunk['usage'])) {
$this->recordUsage($feature, $chunk['model'] ?? null, $chunk['usage']);
}
$text = $chunk['choices'][0]['delta']['content'] ?? '';
if ($text !== '') {
yield $text;
}
}
}
}
}
The buffer is there because a network read can end halfway through an event. The loop only parses up to the last blank line and keeps the rest for the next read. stream_options.include_usage asks the API to send token counts in a final event, which is how streamed calls still get recorded.
Laravel 13 has response()->eventStream() for the other half, sending events on to the browser. It’s covered in the Laravel responses docs:
Route::get('/ai/explain', function (\Illuminate\Http\Request $request, \App\Services\Llm\LlmClient $llm) {
$question = $request->string('q')->limit(500)->toString();
return response()->eventStream(function () use ($llm, $question) {
foreach ($llm->stream([
['role' => 'user', 'content' => $question],
], feature: 'explain') as $text) {
yield $text;
}
});
})->middleware('throttle:10,1');
Here’s what the browser actually receives for a short two-chunk answer, captured from the test app:
event: update
data: Queues
event: update
data: keep requests fast.
event: update
data: </stream>
On the front end, a plain EventSource listening for update events works, and you close it when </stream> arrives. If your UI is React or Next.js, streaming AI responses in a Next.js UI shows the rendering side, including what to do when the user navigates away mid-answer.
Two warnings for this route. Every open stream holds a PHP worker for as long as it runs, so the throttle middleware isn’t optional on a public endpoint. And some proxies and hosts buffer responses, which turns a stream back into one slow response. If chunks arrive all at once in production but not locally, check the proxy before blaming the code.
Tracking tokens and cost per feature
Every response, streamed or not, includes a usage block with prompt and completion tokens. The client writes one row per call:
Schema::create('llm_usages', function (Blueprint $table) {
$table->id();
$table->string('feature')->index();
$table->string('model');
$table->unsignedInteger('input_tokens');
$table->unsignedInteger('output_tokens');
$table->decimal('cost', 12, 6);
$table->timestamps();
});
private function recordUsage(string $feature, ?string $model, array $usage): void
{
$in = (int) ($usage['prompt_tokens'] ?? 0);
$out = (int) ($usage['completion_tokens'] ?? 0);
LlmUsage::create([
'feature' => $feature,
'model' => $model ?? config('services.openai.model'),
'input_tokens' => $in,
'output_tokens' => $out,
'cost' => ($in * config('services.openai.price_in')
+ $out * config('services.openai.price_out')) / 1_000_000,
]);
}
The feature column is the part that pays off. The provider’s dashboard tells you what you spent. This table tells you that ticket summaries cost a little and the explain button costs far more because people paste whole documents into it. That’s the question you’ll actually get asked. Monthly spend per feature is one query:
LlmUsage::selectRaw('feature, sum(input_tokens) as input, sum(output_tokens) as output, sum(cost) as cost')
->where('created_at', '>=', now()->startOfMonth())
->groupBy('feature')
->orderByDesc('cost')
->get();
The stored cost is an estimate built from the prices in your config, so keep those current. Cached input, batch discounts and different prices per model all make the real invoice differ a bit. It’s close enough to spot which feature is growing, which is what you need it for.
If you’re deciding which features deserve an LLM in the first place, my guide to using AI in software development responsibly covers the review and data questions, and Diligent Creators has a plain-language overview of adding AI features to a web app with OpenAI APIs that’s handy for the non-developers in the room.
Testing it without spending a cent
Because everything goes through Laravel’s HTTP client, Http::fake() covers it. No API key, no network, no bill. Here’s the job test, which also checks the cost maths with example prices of 0.15 and 0.60 per million tokens (made-up numbers for the test, not anyone’s real pricing):
public function test_job_saves_summary_and_records_usage(): void
{
Http::fake([
'api.openai.com/*' => Http::response([
'model' => 'gpt-4o-mini',
'choices' => [['message' => ['content' => 'Customer was billed the wrong amount.']]],
'usage' => ['prompt_tokens' => 1200, 'completion_tokens' => 300],
]),
]);
$ticket = TicketSummary::create(['body' => 'My invoice shows the wrong amount.']);
SummarizeTicket::dispatchSync($ticket);
$this->assertSame('Customer was billed the wrong amount.', $ticket->fresh()->summary);
$usage = LlmUsage::sole();
$this->assertSame('ticket-summary', $usage->feature);
$this->assertSame(1200, $usage->input_tokens);
// (1200 * 0.15 + 300 * 0.60) / 1,000,000
$this->assertEqualsWithDelta(0.00036, (float) $usage->cost, 0.0000001);
Http::assertSent(fn ($r) => $r->hasHeader('Authorization', 'Bearer test-key')
&& $r['max_tokens'] === 120);
}
Streaming can be faked the same way. Hand the fake a string of server-sent events and the client reads it like a real stream:
public function test_stream_yields_chunks_and_records_usage(): void
{
$sse = implode('', [
"data: {\"choices\":[{\"delta\":{\"content\":\"Queues \"}}]}\n\n",
"data: {\"choices\":[{\"delta\":{\"content\":\"keep requests fast.\"}}]}\n\n",
"data: {\"model\":\"gpt-4o-mini\",\"choices\":[],\"usage\":{\"prompt_tokens\":20,\"completion_tokens\":5}}\n\n",
"data: [DONE]\n\n",
]);
Http::fake(['api.openai.com/*' => Http::response($sse, 200, ['Content-Type' => 'text/event-stream'])]);
$chunks = iterator_to_array(app(LlmClient::class)->stream(
[['role' => 'user', 'content' => 'Why queues?']], 'explain'), false);
$this->assertSame(['Queues ', 'keep requests fast.'], $chunks);
$this->assertSame(5, LlmUsage::sole()->output_tokens);
Http::assertSent(fn ($r) => $r['stream'] === true
&& $r['stream_options'] === ['include_usage' => true]);
}
The full test file also checks that a 400 fails the job without a retry and that a 429 is rethrown so the queue retries it. All six pass on the versions above. For more on faking external APIs, including sequences and asserting on requests, see testing external APIs in Laravel with Http::fake.
FAQ
Do I need an OpenAI PHP package for Laravel?
No. A package saves some typing, but the API is plain JSON over HTTPS, and Laravel’s HTTP client handles it in a few lines. Going without one also means switching providers is mostly a config change, as long as the new one accepts the same request format.
Should LLM calls be queued or streamed?
Queue them when nobody is watching the result arrive: summaries, classification, drafts for later review. Stream them when a person is waiting on the screen. Making a waiting user poll a queued job gives you the downsides of both.
What timeout should I set?
Long enough for your slowest normal response and no longer. Start by measuring a few real calls for your prompt sizes, then set the HTTP timeout a bit above that and the job $timeout above the HTTP one, so the job can catch the error instead of being killed mid-request.
How do I keep LLM costs under control in Laravel?
Cap max_tokens per feature, throttle public endpoints, skip work that’s already been done, and record usage per feature so you can see where the money goes. Limiting how much text users can send in, like the limit(500) on the explain route, helps too.
Where should the API key live?
In .env on the server, read through config/services.php. Never in front-end code and never committed. The browser talks to your Laravel route, and only Laravel talks to the provider.
I’m Shahrukh Akhtar (Shaz3e), CTO & Co-Founder of Diligent Creators Pvt. Ltd., a Technology Executive & Product Builder. I run client delivery and build SaaS products, including EloSync. You can find me on X, LinkedIn, Peerlist and GitHub.
