What actually breaks when you put an AI agent in production
A demo agent and an agent that runs unattended every day are two different things. The four ways production breaks them, with a concrete fix for each, and what actually keeps them running.
A demo agent and an agent that runs unattended every day are two different products. The demo has a human watching every step, ready to catch anything odd. Production has nobody watching, and it runs while you sleep. Almost everything that makes an agent look easy in a demo is the exact thing that breaks it in production.
I have shipped agents that run on their own, from an SEO auditor to a brand-analysis agent that people trigger on demand. Here are the four failures I keep hitting, and the fix for each.
| Demo agent | Production agent | |
|---|---|---|
| Who is watching | A human at every step | Nobody, it runs while you sleep |
| What it reads | The one tidy example | The messy real world |
| How it fails | Loudly, and you catch it | Quietly, for an hour before anyone notices |
| The hard part | The model | Everything built around it |
1. Silent failures
The worst failure is not a crash. A crash is loud and you notice it. The worst failure is an agent that keeps running and quietly returns nonsense for an hour before anyone spots it. An API changed the shape of its response, a rate limit kicked in mid-run, a prompt drifted, and the agent carried on as if nothing happened.
The fix is reporting first. Before an agent does anything useful, it has to be able to tell you what it did, in a place you actually look. Every run reports its result, and anything unusual is flagged rather than buried in a log nobody reads. My SEO agent posts to Discord on every run for exactly this reason. An agent you cannot see is an agent you cannot trust.
2. Model drift and messy inputs
The model behind the agent changes over time. So do the websites, emails and files it reads. A format that was clean last month arrives malformed today, with a missing field or an extra column. In a demo you feed it the tidy example. In production it meets the real world, and the real world is messy.
The fix is to validate at the edges. The agent checks what comes in and checks what it produced against clear rules, instead of trusting that the input is well formed and the output is sane. When something falls outside the rules, it stops and asks rather than pushing a bad result downstream where it is far more expensive to unwind.
3. Unbounded autonomy
The scariest version of an agent is one with a direct write path to something that matters. It can change your live site, send the email, or move the money, with no gate in between. That is fine right up until the one time it is wrong, and then it is very much not fine.
The fix is a gate before anything irreversible. My SEO agent never writes to production. It proposes every change as a pull request, so a human approves before anything lands. That single decision is what makes it safe to run unattended. Autonomy is not about removing the human, it is about moving the human to the one point where a mistake would actually hurt. I run a sales assistant on the same principle. It drafts the reply and always leaves the send to a human.
4. Cost that quietly runs away
Agents loop. They retry. They pull in more context than you expect. A small bug in a retry path can turn into a large bill overnight, and unlike a crash, nothing tells you until the invoice does. This one stings because it looks like everything is working.
The fix is budget guardrails. Caps on retries, on context size, and on how much a single run is allowed to spend. When the agent hits a limit it should fail closed and report it, not keep spending in the dark.
The pattern underneath all four
Notice that none of these four are about the model. A production agent is maybe twenty percent the model and eighty percent the unglamorous infrastructure around it: reporting, validation, gates and guardrails. That is precisely the part a demo skips, and precisely the part that decides whether the thing survives contact with the real world.
It is also the part I actually build. If you want an agent that runs in production instead of one that demos well, that surrounding layer is the job, and a fixed-scope sprint is how I get it right from the start.
Frequently asked questions
Does this mean AI agents are not ready for production?
No. It means the model is the easy part, and the layer around it is the work. Built with reporting, validation, gates and guardrails, an agent runs unattended and stays reliable.
What is the single most important safeguard?
A gate before anything irreversible. Nothing that cannot be undone, a live change, a sent email, a payment, should happen without a human approving it first.
Can a non-technical founder run one safely?
Yes, when the surrounding layer is built for it. You approve, the agent proposes, and you never babysit code. That is the whole point of moving the human to the one decision that actually matters.
How do you stop the cost from running away?
Hard caps on retries, on context size and on what a single run can spend, with fail-closed behaviour. When the agent hits a limit it stops and reports it instead of spending in the dark.