Calculate when prompt caching pays for itself
Prompt caching saves money when enough eligible prefix tokens are reused before expiry to offset cache-write premiums. Calculate cached-prefix cost separately from uncached suffix and output cost, then validate actual cache usage rather than assuming every repeated request is a hit.
Inside this article 5 sections +
Prompt caching can reduce the cost of repeatedly processing the same input prefix. It does not cache the answer, remove output-token charges, or guarantee that two similar requests share a cache entry. The useful economic question is how many eligible prefix tokens are actually reused and what each cache write costs.
Consider a support assistant with a stable instruction and policy prefix followed by changing account facts and a user question. The stable prefix is a candidate for caching; the changing suffix still needs ordinary processing. Keep those components separate in both prompt design and cost accounting.
Use a dated price basis and a precise workload
Use explicit worksheet inputs: $1 per million ordinary input tokens, $5 per million output tokens, $1.25 per million tokens written to a short-duration cache, $2 for a longer-duration cache write, and $0.10 per million cache-read tokens. These are illustrative rates for comparing policies, not a current vendor quotation. The example excludes discounts, taxes and platform-specific modifiers.
Obtain the current prices and cache rules for the exact model, platform, region and processing tier before budgeting. Providers can differ on automatic versus explicit caching, minimum prefix length, expiry, cache-write charges and retention. Record those inputs with the calculation. A correct formula using another deployment’s prices still produces the wrong estimate.
Assume an illustrative workload of 100 requests, each with a 10,000-token eligible prefix, 1,000 uncached suffix tokens, and 500 output tokens. Without caching, the prefix costs $1, the suffix costs $0.10, and the output costs $0.25. Total model usage is $1.35 for this invented batch.
Calculate hits and writes separately
| Cache-hit share | Prefix reads + writes | Suffix + output | Total |
|---|---|---|---|
| No caching | $1.0000 | $0.3500 | $1.3500 |
| 0% hits | $1.2500 | $0.3500 | $1.6000 |
| 50% hits | $0.6750 | $0.3500 | $1.0250 |
| 80% hits | $0.3300 | $0.3500 | $0.6800 |
| 95% hits | $0.1575 | $0.3500 | $0.5075 |
The no-hit row matters. If a changing timestamp or reordered tool definition invalidates the prefix on every request, the cache-write premium makes this workload more expensive. A repeated business task is not necessarily a repeated byte- or token-compatible prefix under the provider's cache rules.
Use actual reported usage for partial hits. A request may reuse only part of the intended prefix, write another segment, and process a changing suffix normally. A request-level hit percentage can hide that distinction. Token-weighted reuse is usually more informative for the cost calculation.
Derive the break-even reuse rate
Let B be the ordinary input price, W the cache-write price, R the cache-read price, and h the fraction of eligible prefix tokens read from cache. The expected cached-prefix price is hR + (1 − h)W. Caching is cheaper for that prefix when this quantity is below B.
hR + (1 - h)W < B
h > (W - B) / (W - R)
Five-minute example:
h > (1.25 - 1) / (1.25 - 0.10)
h > 0.217391... # about 21.7%
One-hour example:
h > (2 - 1) / (2 - 0.10)
h > 0.526316... # about 52.6%
These thresholds assume the same prefix size and that every miss incurs the stated write price. They exclude engineering effort and any latency value. The one-hour option costs more to write, but may achieve more reuse across widely spaced requests. Compare its observed hit pattern with the five-minute option rather than choosing solely from the threshold.
If output dominates the bill, even excellent prefix reuse may have a modest effect on total cost. Likewise, tool charges and human review remain outside this token worksheet. Carry the measured model saving into the full task-cost model before making a business claim.
Stabilize useful content without crossing data boundaries
Place genuinely stable instructions and reusable context before changing request-specific material where the API's caching structure supports it. Keep serialization deterministic: stable tool order, consistent whitespace handling, and versioned policy text. Do not preserve outdated policy merely to maintain a cache hit.
Separate shared public instructions from tenant-specific or user-specific evidence. Prompt caching does not replace authorization, deletion, or data-handling controls. Verify the provider's cache isolation and retention behavior for the chosen deployment, and ensure each request includes only information the requester may use.
Do not confuse provider prompt caching with an application answer cache. An answer cache can return a prior response without generation and needs its own permission, source-version, and freshness checks. Provider prompt caching reuses computation for supplied input; the application still owns which input is supplied.
Validate economics from production-shaped traffic
Record ordinary input, cache creation, cache read, and output tokens by prompt version and workflow. Include cold starts, long gaps, policy updates, and bursts of concurrent requests. A test that sends the same prompt repeatedly in a few seconds can overstate reuse for a workflow used sporadically throughout the day.
Compare expected costs with provider usage and billing records, keeping pending or missing usage visible. Watch for a sudden rise in writes after a prompt deployment. A harmless-looking dynamic field near the start of the prompt can change the economics without changing answer quality.
Use the AI automation cost model for the full workflow and model-routing economics for cascades. Our AI integration work can connect cache measurements to those operating decisions.