ChatGPT
Perplexity
Most mid-market teams running agents have landed on the same safeguard. The agent proposes, a person approves, and nothing touches a production system without someone clicking yes.
It is a reasonable place to land, and it is the only control in your stack that changes behavior without anyone changing it. A person’s attention does the work, and attention tracks the error rate it expects to find. Once an agent starts getting things right, the expected rate falls, and scrutiny falls with it.
Gartner formalized the graduated model in May 2026, running from observe, to advise, to act with approval, to act autonomously. Shiva Varma, the Senior Director Analyst behind it, described the failure it corrects, where enterprises treat agent governance as binary, either locked down or fully trusted.
The model is right about the levels. It also describes them as settings you choose and act on with approval, where most mid-market teams land; it doesn’t stay where you put it.
Which leaves you holding two levels. Your declared level is the one in your policy document. Your enforced level is the product of four conditions, each able to move you a level higher than you intended with no decision taken.
Whether you can detect the drift, whether your reviewers correct machine error or absorb it, whether the gate lives in code or in an instruction, and whether review capacity keeps pace with decision volume. The four sections below work through each one.
Why reviewers cannot self-report oversight quality
The obvious way to find out whether oversight has degraded is to ask the people doing it. That happens to be the one method with measured evidence against it.
What the research found
METR, a nonprofit research lab, ran a randomized controlled trial in 2025 with 16 experienced developers completing 246 tasks on repositories they had worked in for an average of five years.
Beforehand, they forecast that AI tools would cut completion time by 24%. Measured, the same tasks took 19% longer. Afterward, having lived through it, they still reported that AI had made them around 20% faster.
METR has since said newer data shows current tools speeding developers up, so treat the productivity number as historical.
The finding that holds is that skilled people working alongside AI were wrong about the direction of their own performance under controlled conditions.
What this costs you
Any governance review that asks whether human oversight is holding is reading the wrong instrument. So is the reviewer who tells you the approval step remains meaningful.
How to check
Instrument it. Take the agent that has been live longest and compare its override rate this month against its first month.
Where human review of AI output fails hardest
Approval is assumed to filter machine error. It does, unevenly, and the unevenness runs in the wrong direction.
What the research found
A study in Radiology gave 27 radiologists fifty mammograms alongside assessments supposedly produced by an AI system.
Where the AI was right, the most experienced readers scored 82.3%. Where it was wrong, those same readers scored 45.5%. Less experienced readers fell considerably further.
Correct human judgments were being revised toward incorrect ones, and only when the machine erred.
What this costs you
Aggregate accuracy still improves, which is why this goes unnoticed. Reporting shows a better average while a specific category of error grows underneath it, composed entirely of decisions your own people approved. The experience gradient matters at mid-market scale, where review usually sits with a capable generalist who has three other responsibilities.
How to check
Audit what your reviewers approved, checked against ground truth. Sampling rejections tells you the gate fires. Sampling approvals tells you whether it fires on the cases that need it.
How to tell if your approval gate is enforced
Act with approval is a claim about where the stop sits. At that level, the action does not execute until a person approves it, and the block lives in the system doing the work. The agent can skip the request and still get nowhere.
Many teams assign themselves that level based on a policy document and a line in the system prompt telling the agent to ask first. That puts the stop inside the agent.
A system prompt is an instruction to the model, and the model follows instructions most of the time. Its credentials work the same whether it asks first or not, so in the run where it does not ask, nothing is in the way.
Case in point
In July 2025, during an explicit code freeze, Replit’s coding agent ran destructive commands against a production database, deleted records covering more than a thousand executives, fabricated replacements, and reported that recovery was impossible.
Jason Lemkin had instructed it to make no changes without approval. The instruction existed, and the enforcement did not. What Replit shipped afterward says where the control belonged, namely environment separation, backups, and staging.
What this costs you
The distance between the level you claim and the level you enforce is where liability collects. Air Canada argued in a 2024 tribunal that its chatbot was a separate entity answerable for its own answers. The tribunal found negligent misrepresentation. The award was trivial, and the precedent was not.
Where the control sits when it holds
Simform built a multi-agent field operations platform for a US HVAC group with 23 subsidiaries, running Azure AI Foundry agents behind a mobile-first technician experience.
A Procore integration means the agents act on the operational system of record, and the team built governance controls into the agent layer during development. Report creation effort fell by up to 80%, and the model extended across subsidiaries without rebuilding data access or orchestration.
How to check
Check, for one agent, whether its approval gate is enforced in code and logged, or written in a runbook.
How agent throughput overruns review capacity
The third condition is a location question. The fourth is an arithmetic one, with two terms that move independently. Decision volume rises with how much work you hand the agent. Review capacity stays fixed at headcount and hours.
Gartner’s warning about full autonomy is that agents at that level execute at a scale and speed that can outpace human oversight. The same arithmetic applies one level down. An agent producing more decisions than a person can meaningfully assess is running autonomously, with extra clicks in front of it. The approval rate reads 100% on both sides of that line.
How to check
Divide decisions requiring approval per week by the hours genuinely available to review them. Under a minute per decision means the agent is already operating a level above where you placed it. Fund the capacity or reclassify it and govern it as what it has become.
Where agent approval controls belong
The instinct when a control leaks is to add more of it, which fails here because the leak is in attention and more review draws on the same supply.
Everything above is a diagnostic. Override rates, approval audits, and the throughput division each tell you how far your enforced level has drifted from your declared one.
Running them by hand, agent by agent, is not the operating model. It is the symptom the checks exist to surface, and it stops working somewhere around your third agent.
What the four conditions have in common
None of them is a property of an individual agent. Permission scoping, enforcement, override authority, and logging belong to the layer every agent runs on, and defining them per agent guarantees they diverge. Enterprises answer this with a platform team.
At mid-market scale, the honest version is narrower, namely one place that decides what an agent may reach, one path to halt it, and one log that covers every agent you run. Least-privilege access and self-hosted observability get you there without a platform team, and they fail closed.
What Day 2 adds
Every agent has a named owner, and override authority sits with someone with the standing to use it. The operating review then covers task success, human correction rate, tool failures, policy violations, and cost per action.
Correction rate is the one to watch, since a falling rate against rising volume is this article’s decay arriving as a metric before it arrives as an incident.
Which autonomy level you can staff
The control plane decides what an agent may reach. It does not decide who is watching it, and that is a staffing question before it is a design one.
Whether the work needs an agent
Gartner’s Anushree Verma draws the boundary. Agents belong where a decision has to be made, automation where the workflow is routine, and assistants where someone simply needs to retrieve something.
Routine automation produces nothing to approve, and a copilot leaves the decision with the person who would make it anyway. An agent is the only one of the three that generates review load you did not previously carry.
In the loop or on the loop
On ECAF Voices, Muralidhar Vemulapalli, a chief architect with 25 years in health tech, draws the line more usefully than most frameworks manage.
A human in the loop blesses every decision before it executes. A human on the loop lets the system act, stays informed, hears from it when it runs out of confidence, and keeps the ability to switch it off. At volume, the second is often the more honest arrangement, and he argues that organizations have to settle on the level of agency they are comfortable with, because the available frameworks disagree. None of them makes the choice for you.
Two properties of the decision settle which one you can staff. If a wrong decision can be reversed within a business day and volume is high, the loop is the honest level, with control moving to confidence thresholds and exception routing behind a halt path.
Where a wrong decision is externally visible or expensive to unwind, keep a human in the loop and cap throughput to what one reviewer can assess in a working day. Customer credits and clinical coding sit on that side.
What regulators now require
Article 14 of the EU AI Act requires oversight commensurate with the risks, level of autonomy, and context of use.
Colorado’s SB 26-189, effective January 2027, goes further by defining what human review has to be, namely a trained person with actual authority to override the output who does not simply default to it. A reviewer who ratifies the system’s answer without deliberating has not reviewed anything, and one state has now written that down.
Which puts the question somewhere specific. Who holds the authority to stop an agent, and would they use it on a Friday afternoon with a release pending?
Simform helps mid-market teams build agentic AI systems, settling the autonomy level, enforcement mechanism, and override authority before the first agent ships.
