Building Production-Ready AI Agents
Why an agent run belongs in a background job, what a token budget actually costs, and the four guardrails I put in front of every model call.
An agent is a background job, not a request
The first thing that goes wrong when people put an agent into a product is the shape of the execution. A single agent run takes minutes. It calls a model, reads files, calls the model again, and writes something. That is not a web request.
In Lemniscate the agent runs as a job in a Redis and BullMQ worker. It reads a repository, proposes changes, writes the code, pushes a branch and opens a pull request. The browser never holds the connection open waiting for the result; it watches a log stream over server-sent events and can be closed and reopened without killing the run.
The second thing that goes wrong is trusting a single pass. Lemniscate optionally sends its own diff to a second model call before anything is merged. The reviewer is cheaper than the author and catches the obvious mistakes.
The token budget is a real budget
My most expensive lesson came from a reasoning model. It bills its chain of thought against the same output allowance as the answer. On a long reply the reasoning consumed the entire budget, and the model returned nothing at all: no answer, no error, just an empty string.
I measured it rather than guessing, raised the ceiling, and added continuation handling that asks the model to resume from the exact point it stopped and forbids repeating what it already wrote. Since then the cost of a long answer is a number I can predict instead of a surprise.
Every integration I build now has four things in front of the model call: a timeout, a schema check on the output, a hard token ceiling, and a fallback path.
Fallbacks are not paranoia
The AI doctor on this site answers questions about my health data through tool calls. When the hosted model is unavailable it falls back to a model running on the same server, slower and less capable, but present. When the API returns malformed output, the assistant answers with what it can verify instead of inventing a number.
That is the difference between a feature and a demo. A demo assumes the provider is up, the response is well formed, and the user is patient.
Make the run auditable
An agent that changes code has to leave a trail. In Touchstone the agent-facing API returns versioned report schemas (report/v1), so a client can tell what shape of answer it received. Every step is recorded: what was asked, what the model returned, what was accepted.
Without that trail you cannot answer the only question that matters after an incident: what did it actually do.
The honest summary
A production agent is mostly ordinary distributed-systems work: queues, retries, idempotency, budgets, logs, fallbacks. The model is one dependency inside that machine, and it behaves like a dependency with unpredictable latency, real cost, and a habit of being confidently wrong.
Treat it that way and the features hold up. Treat it as magic and you ship something that works exactly once, on your machine, during the demo.