Meterwise guide / 21 November 2026
Prompt caching: how to stop paying full price for repeated context
What prompt caching is
Many applications send the same instructions, policy documents or reference material with every request. Without caching, the provider processes that context as new input each time and charges the normal input price. Prompt caching lets the provider reuse suitable repeated context and bill it at the cached-input rate.
On current Claude models, cached input costs 2.5% to 10% of the normal input price. On the OpenAI models in our table, it costs 5% to 10%. The discount applies to cached input, not to the response. Output tokens continue to use the model's normal output price.
When caching pays off
Caching is useful when requests share a substantial, stable prefix. Examples include a common set of operating instructions, the same product catalogue used for classification, or a document discussed over several requests. The repeated context must be large enough and reused often enough for the saving to matter in your own workload.
It is less useful when every request contains different material, or when the shared part is small compared with the changing input and output. Review actual token volumes rather than assuming that a long system prompt automatically creates a worthwhile saving. The calculator can show the cost effect for the cached share you observe.
Put the stable part first
Structure each prompt so the content that stays the same comes first. Place shared instructions and reference material before the user question, retrieved passages or other request-specific content. A change near the start can prevent the rest of the context from matching an earlier request.
Keep formatting, ordering and wording consistent across requests. Avoid inserting timestamps, request identifiers or changing metadata into the stable section. If different tasks need different instructions, group traffic by prompt type instead of combining every possible instruction into one variable block. Always follow the provider's current caching requirements for the model and API you use.
Check the cache hit rate in usage data
Do not treat enabling caching as proof that it works. Export usage by model and prompt type, then compare ordinary input tokens with cache-read input tokens. The cache hit rate should describe the share of eligible input actually read from cache, not simply the number of requests that included a cache marker.
Segment the data before drawing conclusions. A healthy overall rate can hide a workflow whose changing prefix never matches. Compare the observed cached share with what you expected from the prompt design, and investigate workflows with repeated misses. Recheck after prompt changes, releases and shifts in traffic mix.
Common mistakes
The most common problem is placing variable content before the reusable context. Other causes include rewriting stable instructions on every request, splitting similar traffic across many prompt variants and measuring requests instead of tokens. A high request count does not necessarily mean a high volume of cached input.
Also check cost and quality together. Caching does not remove the need to choose an appropriate model or control output length. Test a representative sample of your own requests after changing prompt structure, and confirm that responses remain suitable. For a complete estimate, include cache write behaviour from your provider's current price list; the calculator on this site ignores cache write surcharges.
Keep a record of which prompt version produced each usage pattern. When the hit rate changes, that record helps you distinguish a prompt edit from a shift in customer traffic. Review the stable prefix and the usage export together, then correct the structure before counting any additional saving. This also gives your team a clear way to maintain caching as the application evolves.