ChatGPT
Perplexity
Shipping an AI agent comes with a satisfying sense of finality. The pilot cleared its evaluation, the demo landed, and the team rolled off to the next build. That sense of finality is the trap.
The launch is the moment the agent starts doing the job for real, on live users and inputs nobody rehearsed, and it is also the moment most teams quietly stop watching it closely.
Microsoft’s own Foundry team is direct about where the real difficulty in an agent program lies: in the long run, keeping it reliable after it goes live. That work is a discipline with its own cost and a real owner, and most mid-market teams have planned for neither. The return on the whole investment tends to leak out of that blind spot.
An agent that passed at launch degrades quietly in production
When traditional software breaks, it announces itself with errors and alerts, and someone gets paged within minutes.
An agent rarely fails that cleanly. It keeps responding in full sentences, with the same confident tone it had on day one, even as the quality of its answers slides underneath that surface. The causes are ordinary. Nobody touches the agent, but the systems underneath it keep moving.
A model gets upgraded beneath it, or the data it retrieves slowly goes stale, and the behavior you validated drifts without a single line of code changing. None of that surfaces as an error. The agent keeps answering, and the answers just get quietly worse.
This is not an edge case. DORA’s 2025 report, drawn from nearly 5,000 engineers, found that AI mostly amplifies what a team already is, sharpening strong operations and magnifying weak ones.
The uncomfortable implication is that the teams least able to notice an agent drifting are exactly the ones it happens to most. What matters more than the launch result is whether you would catch it, three months in, when the agent has quietly stopped doing the job you deployed it for.
Stay updated with Simform’s weekly insights.
Few teams can stop an agent once it starts misbehaving
Instrumentation is the part most teams got right. Dashboards track latency, token spend, and traces, reliably showing you that something has shifted.
The harder half, mostly still missing, is acting on it in the moment, holding an agent inside the actions it is permitted to take and cutting it off when it steps past them.
We saw this with a support agent who handled refund requests cleanly at launch and, after the underlying model was upgraded, began approving exceptions that it should have escalated.
The dashboards showed nothing alarming: every response still read as reasonable, the volume looked normal, and the drift ran for weeks before anyone connected the rise in refunds to the agent.
Visibility caught nothing because visibility was never the control. Deloitte’s 2026 research across 3,235 leaders puts a number on the gap.
Only 21% report a mature governance model for agentic AI, and the roughly four in five who do not are missing the basics it entails, including the rules governing which decisions an agent may make on its own and the live monitoring that flags when it wanders.
McKinsey’s 2026 work finds the same imbalance in every region it studied. It describes the job ahead as moving governance from written policy into enforcement that runs at the instant the agent acts.
What to do about this
Take your highest-stakes agent and write down, in plain language, the actions it may complete unattended and those that require a human to sign off.
Put one hard limit behind that line, a spending ceiling or a kill switch, before the next agent goes live. An agent you can watch but cannot halt grows more dangerous with every one you add.
Operating an agent is a defined practice with named parts
Running an agent well follows a defined practice, and the major vendors now describe its parts in the same way. IBM’s lifecycle guidance sets out continuous production monitoring for performance, drift, and operational risk, with versioning and rollback so that a change that degrades quality can be undone.
Microsoft’s Foundry observability lays out the same runtime loop in operational terms: evaluating a sampled slice of live traffic, running scheduled checks that detect drift against a baseline, and raising an Azure Monitor alert when output quality drops below a threshold you set.
Cost rides inside that loop, because token-based spend behaves nothing like the steady cost of uptime. It is one reason 98% of FinOps practitioners now manage AI spend and increasingly insist on seeing it down to the token and the model.
Continuous evaluation has a price of its own, since every sampled check spends evaluator compute, so the sampling rate is a dial you tune against the regressions you can afford to miss.
The concrete move
Turn on continuous evaluation for one production agent at a five to ten percent sample, put a quality threshold behind an alert, and show cost per useful outcome on the same screen as that quality number.
Read together, a drop in quality or a spike in cost reaches the same person in the same review, which is how operations stop being a quarterly fire drill.
A lean team should run this layer on the tooling it already has
The reflex at a 200 to 2,000-person company is to hand off monitoring and controls to the engineering team and have them build it in-house. The economics rarely support that.
Continuous evaluation and runtime guardrails are a standing operational load, and a company this size has little spare capacity to staff a function that has to run every day an agent is live.
IDC’s research reports that smaller firms are already meeting needs like this by sourcing AI through as-a-service models and cloud marketplaces instead of carrying the infrastructure themselves.
The practical path is to operate on tooling that already does the continuous work and to build only what it genuinely lacks. Azure AI Foundry runs the same evaluators in development, in CI/CD, and against live traffic, and routes incidents into the on-call rotation a team already staffs, so a small group can set a sampling rate without standing up a separate monitoring team.
The concrete move
Name one owner for agent operations before you approve any build, then list what your current platform already gives you, evaluators and incident routing, and scope the build down to the gaps that remain. Most teams find that gap far narrower than the platform they were about to commission.
It is tempting to read the fact that most AI spending has not reached earnings as a model problem and to wait for better models. McKinsey’s research points the other way.
Nearly eight in ten companies report using generative AI, and about the same share report no material impact on the bottom line. Much of that value does not go missing inside the model. It drains away in the quiet months after launch, from agents that work on the day they ship and from nobody operating six weeks later.
The question worth carrying into your next review is easy to ask and uncomfortable to answer. Of the agents already living in your production, which can you prove are still performing at the level you signed off on, and who is accountable the day one of them drifts?
Simform helps mid-market teams operate agents in production with Agentic AI services, so the continuous work of keeping an agent reliable does not land on a team that was never staffed for it.
