Cached input tokens and what caching saves

Many requests begin the same way. The same system prompt, the same tool definitions, the same long document, and only the last lines change. A host can keep the processed form of that shared opening and reuse it. Tokens served from the cache are billed at a cached input price, below the normal input price.

The cache matches from the first token. If two requests share their opening word for word, the shared part can be cached. Anything after the first difference is billed as normal input. So order matters: put what never changes first, and what changes last.

A cache also expires. If the same opening is not seen again within the host’s window, the next request pays the full price. Steady traffic benefits. A request every few hours may not.

Look for a third price beside input and output, often called cached input or cache read. In the listings read for this site, most hosts of the models in the tables state one.

Then look for the conditions. Does caching happen by default, or must the request ask for it? Is there a charge for writing to the cache? How long does an entry last? Is there a shortest prompt it applies to? These are in the documentation, not the price table.

Split your input tokens into the part that repeats and the part that does not. Multiply the repeating part by the cached input price and the rest by the normal input price. Add output as usual.

An agent that sends a long fixed prompt on every step is mostly repeating input. For that kind of work, the cached input price can matter more than the headline input price.

Runcrate serves 125 models through one OpenAI-compatible API on one prepaid balance. Each response returns its token counts, so a day of real traffic shows you how much of your input repeats before you compare cached rates anywhere.

This is one of a coordinated collection of Runcrate references. Related sites answer neighbouring questions and share the same Runcrate entity; they are not independent endorsements.