Why AI Agent Costs Compound Instead of Add

A chat endpoint costs roughly what you'd expect: requests times average tokens. An agent does not. AI agent cost compounds because each step feeds the accumulated context back into the next one — so spend grows with traffic *times loop depth*, and loop depth depends on the input.

The compounding mechanic

Consider an agent that reads a message, calls a tool, reads the result, and responds. Each model call includes the full running transcript: the original request, every tool call, and every tool result so far. So the input tokens on step N include everything from steps 1 through N-1.

• Step 1 pays for the prompt.
• Step 2 pays for the prompt + step 1's output + the tool result.
• Step 5 pays for all of that, again, plus four more layers.

Why it hides in testing

Prototypes don't loop. Your test inputs are clean, the agent resolves in two steps, and the cost looks fine. Production is messier: an ambiguous request, a tool that returns a wall of text, a model that second-guesses itself. One user with a weird input is all it takes to turn a bounded task into a ten-step spiral.

How to bound agent cost

1. 01Cap the step count. A hard maximum on loop iterations stops infinite spirals dead.
2. 02Trim the context. Summarise or drop old tool results instead of resending them every step — see token usage limits.
3. 03Cap total spend per run. Budget a single agent invocation, not just the account, so one task can't run away.
4. 04Alert on outliers. A run that takes 12 steps when the median is 3 should page you.

const guard = new Guard(anthropic, {
  project: "research-agent",
  // Bound a single run, not just the monthly total.
  cap: { perRun: 0.5, maxSteps: 8 },
});

The mental model to keep: with agents, cost is a function of *behaviour*, not request volume. Bounding behaviour — steps, context size, per-run spend — is what keeps it predictable.