Prompt caching: stop paying full price for the context you send every time

Why repeated context costs so much

An enterprise agent rarely sends a short prompt. Each request usually carries a system instruction, tool definitions, policy documents, a few examples and the conversation so far. Most of that is identical from one call to the next, but without caching the model reads it, and you pay for it, every single time.

Caching changes that. The platform keeps the processed form of a prompt prefix it has seen recently. When a new request starts with the same prefix, those tokens are served from the cache at a much lower price, and usually with lower latency.

Two ways to cache on Google Cloud

Gemini models on Gemini Enterprise Agent Platform (formerly Vertex AI) support two kinds of context caching:

• Implicit caching is on by default for Google Cloud projects. You change nothing in your code: when a request shares a long enough prefix with a recent one, the discount is applied automatically. There is no storage charge, but hits are not guaranteed, and cached prefixes are evicted within hours.
• Explicit caching is a cache you create through the API. You send the content once, set a time to live, and reference the cache by name in later requests. Hits are guaranteed while it lives, and you pay a storage charge for the time it is kept.

At the time of writing, Google's documentation describes a 90% discount on cached input tokens for Gemini 2.5 and later models, standard input pricing for the tokens that create a cache, and a minimum prompt size of a few thousand tokens that depends on the model. These numbers change, so check the current documentation and pricing page before building a business case on them.

Design prompts that hit the cache

Caches match on the beginning of the prompt. One changed character early on, and everything after it is a miss. So the order of your prompt matters more than its length.

• Stable content first: the system instruction, tool and function declarations, reference documents and few-shot examples.
• Variable content last: the user's question, retrieved passages that change per request, timestamps and IDs.
• Keep the prefix byte-for-byte identical. Don't inject the current date, a request ID or a reordered tool list into the system instruction.
• Version your prompts. Changing the stable prefix invalidates every cache built on the old version, so roll prompt changes out deliberately.
• Keep similar traffic together. Implicit caches are short-lived, so requests that share a prefix and arrive close together hit more often.

When explicit caching pays off

Explicit caching suits a large body of context that many requests reuse within a known window: a contract set during a review session, a codebase an agent works on all afternoon, or the product catalogue behind a support agent. Compare the storage cost for the time to live with the input savings from the requests you expect in that window. If only a handful of requests will use it, implicit caching is usually enough.

Measure it

Don't assume the cache is working; read it from the response. Gemini responses report cachedContentTokenCount in their usage metadata, next to the prompt and output token counts. Log those fields for every request, for example to BigQuery, and track the hit ratio (cached tokens divided by prompt tokens) per agent and per prompt version. A sudden drop usually means something variable has crept into the prefix.

response = client.models.generate_content(model=MODEL, contents=contents)
usage = response.usage_metadata

log_row = {
    "agent": "support-agent",
    "prompt_version": PROMPT_VERSION,
    "prompt_tokens": usage.prompt_token_count,
    "cached_tokens": usage.cached_content_token_count or 0,
    "output_tokens": usage.candidates_token_count,
}

Other models on the platform

Partner models have their own caching features. Claude models, for example, use explicit cache breakpoints in the request. The rules are the same: stable prefix first, variable content last, and measure the hit rate from the usage data.

Where to start

Pick your highest-volume agent, measure how much of each prompt is stable, reorder it so the stable part comes first, and watch the cached token count for a week. It is often the cheapest cost reduction available, because it doesn't change what the model sees, only what you pay for it. For the wider picture, read Tokenomics: the unit economics of AI agents.