Anatomy of a $10k LLM Bill: A Post-Mortem

The most useful way to understand an LLM bill spike is to walk through one. This is a composite post-mortem of a failure pattern we see constantly: a $300/month feature that quietly became a $10,000 one, and stayed that way for three weeks because nothing was watching.

The setup

A B2B app had a document-summarisation feature on Sonnet, costing about $300/month across a few hundred users. Stable for months. Then two changes landed in the same sprint - neither reviewed for cost.

The two changes

1. 01A prompt tweak added the full document *and* a set of few-shot examples to every request, roughly tripling input tokens per call.
2. 02A retry wrapper was added for reliability: on any error, retry up to 5 times. It re-sent the now-tripled prompt each time.

Why nobody noticed for three weeks

• The provider console showed a monthly total, and month-to-date looked "a bit high" but not alarming until it wasn't.
• There were no per-feature baselines, so the summariser's 15x jump was invisible against total account spend.
• The invoice - the only real signal - arrives ~30 days late. By the time it landed, three weeks of overspend were already paid.

What would have caught it

1. 01A per-feature anomaly alert would have paged someone the day input tokens tripled, not three weeks later. See budget alerts.
2. 02A per-run cap would have bounded the retry storm - 5x re-sends can't happen if a single run is budgeted.
3. 03A hard monthly cap on the feature would have stopped calls at, say, $1,000 instead of letting them run to $10,000.
4. 04Cost review in code review - a one-line note that this change multiplies token volume.

None of these are exotic. They're the exact playbook in how to stop your bill from spiking: attribute, cap, alert. The $10k lesson is that any one of them, in place beforehand, turns a five-figure surprise into a Tuesday-morning Slack ping.