Operations · Governance · 5 min read
Human in the loop that actually works
Approval fatigue kills most oversight designs. Here is how we decide what needs approval, what needs review, and what should just run.
In short
How should approvals be designed for AI agents?
Require approval only for actions that are irreversible or externally visible, sample-review the rest, and log everything. If people are approving more than a handful of items a day, the design is wrong and they will start rubber-stamping.
Every governance conversation starts with the same instinct: a human should approve everything the agent does. It sounds safe. In practice it produces a queue nobody reads carefully, and the moment the agent makes a mistake, the human who clicked approve is blamed.
Three tiers
- Approve: sending money, contacting a customer for the first time, changing a record of legal significance, deleting anything. A named person decides, in the tool they already use.
- Review: drafts, classifications, internal updates. The agent proceeds; a percentage is sampled for review, and the sample rate rises automatically when quality dips.
- Run: retrieval, summarisation, routing, status updates. Logged, monitored, never queued.
Make approval cheap and informative
An approval request should show what the agent wants to do, why, what it read to decide, and what happens if nothing is done. It should be answerable in one tap from email, Slack or Teams. If the approver needs to open another system to decide, the request is incomplete.
Close the loop
Every rejection is training data. Velum's runtime captures rejections and edits and feeds them into the evaluation suite, so that the same class of mistake is caught before the next release. Over a quarter, approval volume should fall while quality rises. If it does not, that is a signal worth investigating.
Talk this through with us.
Thirty minutes with a Velum lead, on one function from your operation.