Tokenomics: the unit economics of AI agents

Price per token is not cost per task

Model pricing is quoted per million tokens, split into input tokens (what you send) and output tokens (what the model writes), and output usually costs several times more than input. Those prices matter, but they do not tell you what an agent costs. One user request can mean several model calls, tool calls, retrieved documents and a growing conversation resent on every turn. The number the business cares about is cost per completed task.

Where the tokens go

• System instruction and policies, sent on every call.
• Tool declarations. Every tool the agent could use is described in the prompt, whether it is used or not.
• Retrieved context from search, documents or a lakehouse query.
• Conversation history, which grows each turn because every call resends what came before.
• Reasoning tokens on models that think before answering, usually billed as output.
• The answer itself.

In multi-step agents, history dominates. If every step adds a similar amount of context and the full history is resent at each step, total input tokens grow roughly with the square of the number of steps, not in a straight line.

A simple model

Estimate cost per task by adding up every model call in the task:

cost per task = sum over calls of (
                  uncached input tokens x input price
                + cached input tokens   x cached input price
                + output tokens         x output price )
              + tool and retrieval costs

monthly cost  = cost per task x tasks per month

Build the estimate from token counts logged on real traffic rather than guesses. Real users ask longer questions, trigger more tool calls and retry more often than test scripts do.

A worked example

Take a support agent that finishes a typical task in four model calls. Its system instruction and tool declarations come to 6,000 tokens, each step adds about 1,500 tokens of new context, and each call writes about 400 tokens. The numbers are illustrative; the shape is what matters.

Of those 41,400 input tokens, 24,000 are the same 6,000-token prefix sent four times. With a good cache hit rate, most of the biggest line item is billed at the cached rate. Summarising or trimming history between steps stops the later calls from growing. Neither change affects the answer the user sees.

Levers that move the number

• Route by difficulty. Send routine steps to a smaller, faster model and keep the largest models for the steps that need them.
• Cache the stable prefix. See Prompt caching.
• Load only the tools a step needs. Group tools by task so the agent is not sent every declaration on every call.
• Manage history. Summarise or drop old turns and tool outputs the agent no longer needs.
• Cap output. Set a maximum output length and ask for concise, structured answers; output is the most expensive token.
• Batch what can wait. Batch inference is priced below online requests for work such as nightly classification or document extraction.
• Match capacity to traffic. Pay-as-you-go suits variable demand; Provisioned Throughput suits steady, high volume that needs guaranteed capacity.

Measure cost per task in production

Log the usage metadata of every model call with a task ID, the agent version and the model, and land it in BigQuery next to your business events. Then you can report cost per resolved ticket, per processed invoice or per generated report, and see the effect of every prompt or model change. Add a budget alert per agent, so a looping agent or a prompt regression shows up within hours rather than on the monthly bill.

The takeaway

Treat tokens like any other cloud resource: estimate before you build, measure in production, and give every agent an owner and a budget. Teams that do this can scale agents with confidence. Teams that do not tend to find out from the invoice.