The Multilingual Cache Tax: How a token-count cache threshold turns cross-lingual token inflation into an escapable, provider-dependent pricing artifact

Abstract

Commercial LLM APIs bill a cached prompt prefix at a fraction of the normal input rate, but only once the prefix exceeds a fixed token count (a floor, commonly 1,024 tokens). Because the same content costs a different number of tokens in different languages, this token-denominated floor treats languages differently: an English prompt, being the least inflated, reaches the floor later than its translations. Across twelve languages and four tokenizer families, English reaches the floor last, and on live OpenAI billing we confirm content depths at which English is the only one of five tested languages that fails to cache, so a naive English deployment can pay several times more than the identical service in another language. We are careful about what this is. It is not a structural property of language cost. An operator can pad a prompt to the floor, and across 500 tasks we find that doing so is essentially free: a paired analysis bounds any accuracy cost of junk padding below 1.1 percentage points at 95% confidence (82.0% vs 81.8%), including on a context-dependent subtask where distraction was possible, while padding with worked examples instead raises accuracy (86.0%, p=0.001, an effect we attribute to format pinning rather than to caching), and both make the prompt cacheable. In fact escapability is structural: the same inequality that creates the near-floor penalty (discount times inflation below one) is the condition under which padding out of it is profitable. The floor is therefore an arbitrary pricing artifact whose cost falls on operators who are unaware of it. We quantify the artifact across four tokenizer families, on both benchmark prose and real system prompts (where English scaffolding dilutes inflation by 36 to 54%), across two providers (OpenAI and Gemini, where a higher floor widens the gap). We close with a one-line policy fix that eliminates the artifact. Version 3 supersedes version 1: it withdraws the earlier non-monotonic-bias framing in favour of the escapable pricing-artifact framing, adds the r-times-i inequality, a 500-task padding experiment with paired statistics, real-system-prompt measurements, provider re-verification (August 2026), and a real price sheet. Code, data, and live-API logs are included.