Practical notes on the three cost drivers we check in every audit: repeated context, work that can wait, and model choice.
01
Prompt caching: how to stop paying full price for repeated context
Prompt caching lowers the price of context you send repeatedly. Keep the stable part of each request unchanged and at the beginning, then use provider usage data to confirm that requests actually read from the cache.
Batch processing gives 50% off input and output at both providers in the price table. It fits work with a flexible completion time, not live conversations. Start by measuring which token volumes can tolerate the 24-hour processing window.
Model prices vary widely, but the cheapest option is not automatically the right one. Route by task, compare answers on a sample of real requests, and judge changes by cost per completed unit of work as well as quality.