When LLM spend rises, the first question is always the same: *where did it come from?* If the only data you have is the provider's invoice, the honest answer is "somewhere." The fix is cheap and boring: attach a small set of attributes to every model call at the moment you make it, and write them next to the token counts.

What to record on every call

You do not need a data warehouse to start. You need one structured log line or database row per call with these fields:

  • feature: a stable name for the product surface that triggered the call (ticket_summary, search_rerank), not the function name, which changes in refactors.
  • prompt_version: the identifier of the prompt template that was rendered. Without it you cannot tell whether a cost jump came from a prompt edit.
  • model: the exact model identifier requested, plus the one the response reports.
  • tenant or user bucket: enough to spot one account driving a spike. A hashed ID is fine.
  • input_tokens, output_tokens, cached_tokens: as reported by the API response, never estimated by you after the fact.
  • outcome: success, retry, timeout, parse failure, discarded. This is the field people forget, and it matters most.
  • request_id of the parent task, so multiple calls in one agent run can be grouped.

Put this in a thin wrapper around your client so individual call sites cannot forget it. If a call site can skip attribution, someone eventually will, and that call will be the expensive one.

Questions this lets you answer

Once the data exists, several vague worries become simple queries:

  1. Cost per successful task, by feature. Divide total spend for a feature by the number of tasks that actually reached a usable result. Retries, discarded outputs, and parse failures all land in the numerator, so the figure reflects what a working result really costs.
  2. Which prompt version changed the average input size? Group by prompt_version and compare token distributions. A new instruction block that adds 800 tokens to every call is easy to miss in review and obvious in this table.
  3. Is the cost concentrated? Sort tenants by spend. If one tenant accounts for a large share, you have a product or pricing question, not an engineering one.
  4. How much is retry overhead? Count calls whose outcome was a retry or discard. If it is a meaningful fraction of tokens, fixing a flaky output format may save more than switching models.

Two traps

First, attribute at the *task* level as well as the call level. An agent that makes eleven calls to answer one question looks cheap per call and expensive per task; only the parent ID reveals it.

Second, do not let attribution data contain prompt text or user content. Store identifiers and counts. If you later need examples, sample them separately under your normal data-handling rules.

The closing point

Cost control is mostly a visibility problem before it is an optimization problem. Teams that cap budgets without attribution tend to throttle the wrong thing. Spend an afternoon on the wrapper, and the next time the bill jumps, you will be reading a table instead of guessing.

AI IntegrationDeveloper ToolsSoftware Engineering