An agent is a loop: the model reads context, chooses a tool, observes the result and decides what to do next. That loop is powerful, and it fails in ways ordinary software does not — it can call the wrong tool with plausible arguments, follow instructions hidden in a document, or spin through dozens of steps on a single request. Production readiness means engineering around those failure modes.
Tools are API contracts
Treat every tool like a public endpoint. Define its arguments with a schema, validate the model's output against it, and reject anything that does not parse.
const tools = {
lookup_order: {
schema: z.object({ orderId: z.string().regex(/^ord_[a-z0-9]+$/) }),
run: (args: { orderId: string }, viewer: Viewer) => orders.getForViewer(viewer, args.orderId),
},
request_refund: {
schema: z.object({ orderId: z.string(), reason: z.string().max(500) }),
run: (args: { orderId: string; reason: string }, viewer: Viewer) => approvals.queue('refund', viewer, args), // human approves
},
};
Least privilege, inherited from the user
The agent should act with the permissions of the person it is helping, never with a service account that can see everything. If the user cannot open an order in the UI, the tool must not return it to the model either.
Humans approve irreversible actions
Reading is cheap to get wrong; refunds, emails and deletions are not. Our default is that any action with external side effects becomes a proposal that a human confirms, with the agent's reasoning attached. Teams relax this per action once evaluation data justifies it.
Treat retrieved content as data
Prompt injection arrives through documents, tickets and web pages the agent reads. Wrap untrusted content in clearly delimited data blocks, instruct the model never to follow instructions inside them, and — more importantly — make sure the tools themselves cannot do damage even if the model is tricked.
Bound the loop
const MAX_STEPS = 8;
for (let step = 0; step < MAX_STEPS; step++) {
const res = await model.generate({ messages, tools: toolSpecs, signal: AbortSignal.timeout(30_000) });
if (!res.toolCall) return res.text;
const tool = tools[res.toolCall.name as keyof typeof tools];
const args = tool?.schema.safeParse(res.toolCall.args);
if (!tool || !args?.success) { messages.push(toolError('Invalid tool call')); continue; }
messages.push(toolResult(await tool.run(args.data as never, viewer)));
}
return escalateToHuman(messages); // never loop forever
Evaluate like you test
- Build a scenario set from real conversations: expected tool calls, forbidden tool calls and acceptable answers.
- Score every prompt, model or tool change against it in CI before it ships.
- Trace every production run (inputs, tool calls, tokens, latency) so failures can be replayed.
Control cost per tenant
Log tokens per request and per tenant, set budgets with alerts, cache deterministic sub-steps, and route simple requests to smaller models. Cost surprises are an engineering problem, not a finance one.
Key takeaways
- Validate every tool call against a schema.
- Run agents with the user's permissions, not a superuser's.
- Put a human in front of irreversible actions.
- Assume injected instructions will arrive; design tools so they cannot cause harm.
- Bound loops, evaluate continuously, and budget tokens per tenant.
