We are early: this is the design we build to and the reasoning behind it, not a retrospective on a fleet of live deployments.
An agent with tool access changes state in systems of record. It moves money, alters entitlements, cancels services. Here is what we consider settled about building one, and why.
Tool design matters more than model choice
The easy quarter to waste is the one spent arguing about models. The contracts are where the behaviour actually comes from.
An agent's competence is bounded by the tools it has. A tool that takes fifteen loosely-typed parameters and returns an untyped blob will produce bad behaviour from any model. A tool with three typed parameters, a clear name, an explicit failure mode and a docstring written for a reader who has never seen the system will produce good behaviour from a surprisingly small one.
Specific things that make the difference:
- One tool, one verb. "Manage subscription" became four tools. Error rates on each dropped sharply.
- Return what changed, not just success. An agent that sees the resulting state can verify its own work.
- Make failures explicit and typed. "Insufficient permission" and "record not found" need different recoveries. A generic error string means the agent guesses.
- Never expose a general-purpose escape hatch. A "run arbitrary query" tool added for flexibility will be used for exactly the things you did not want, and it destroys the property that makes the agent governable — that someone can read the tool register and enumerate what it can do.
Approval queues rot, quietly
The responsible-looking design puts a human in front of a lot of actions. It degrades on a predictable schedule: volume outruns attention, reviewers start batch-approving, and median review time falls below the time it takes to read the case. The control stays on the org chart and stops existing in practice.
This is well documented in every other domain that has automated a review function, and there is no reason to expect agents to be the exception.
What works instead: put humans where their judgement changes the outcome — the policy boundary, disagreement between evidence lanes, vulnerable parties — and add a blind random sample of the autonomous population re-worked independently.
The blind sample is the control that actually tells you something. A rubber-stamped queue tells you nothing while producing a comforting audit trail.
Escalation rate is a floor, not a target
Everyone's first instinct, including ours, is to drive escalation down. It looks like inefficiency.
Watch what happens when you succeed. The agent becomes reluctant to hand over, which means it handles cases it should not, which means the failures move from "handed to a human unnecessarily" — cheap — to "resolved incorrectly with confidence" — expensive and slow to detect.
So we treat the escalation rate as a floor to protect rather than a target to beat. When it drifts down, that is the thing to investigate, not the thing to celebrate.
Memory needs a retention policy, on day one
Agents accumulate context. Left alone, an agent will carry a customer's irritated message from four months ago into today's conversation about a delivery date.
Decide what persists, for how long, and what is scoped to a single interaction. Then decide who can see it, because agent memory is personal data and it will be subject to access requests.
Decide it before launch. Retrofitting a retention policy onto accumulated agent memory is a data project, not a config change.
Traces are the product
Tracing gets built for debugging and turns out to be the most-used part of the system.
Compliance answers regulators from it. Operations works out why a class of case behaves oddly. Product finds friction. And the traces are the highest-quality training data the organisation has, because every one is a real decision with real evidence and a known outcome.
Build the trace store first, retain it like a transaction record, and make it queryable by people who are not engineers.
The estimate everyone gets wrong
How long the integration work takes. Optimism about connecting to systems of record is close to universal, and it is consistently the critical path. The model is a weekend. The connector to the thirty-year-old policy administration system is a quarter.
- Agents
- Production
- Lessons