Guardrails for autonomous AI agents
Agents that call tools, spend tokens and act on systems need budgets, permissions and audit trails before they reach production — not after.
An assistant answers a question. An agent takes actions in a loop. That difference changes the risk profile completely: cost becomes unbounded, and so does blast radius.
Three failure modes we see repeatedly
1. Runaway loops. An agent retries a failing tool call hundreds of times overnight and turns a small task into a four-figure invoice.
2. Over-broad permissions. The agent is given the same credentials as the engineer who built it, so a reasoning error becomes a production incident.
3. No trace. When something goes wrong there is no record of which prompt, which tool call and which model version produced the action.
The guardrails that matter
- A hard token and cost ceiling per run, per agent and per day, enforced outside the agent's own logic
- Step limits and loop detection, with automatic termination
- Scoped, short-lived credentials per agent, with write access granted tool by tool
- Human approval for irreversible actions: payments, deletions, external communication
- Full traces of prompts, tool calls, outputs and model versions, retained and searchable
Treat agents as software, not as prompts
They need code review, staged rollout, monitoring and an owner on call. A prompt change that alters behaviour in production is a deployment, and should follow the same path.
Governance is what makes agents shippable
Teams that put limits and traces in place early ship more agents, not fewer — because the downside of each experiment is bounded and understood.