We padded a prompt with junk to make it cheaper

Once you know the cache discount has a floor, the fix writes itself: if the prefix is too short to cache, make it longer. It sounds like gaming the meter, and the arithmetic says it is not marginal.

When padding wins, exactly

Let F be the floor, r the cached-rate multiplier and p your current prefix length. Padding up to the floor costs F·r per call instead of p. So padding wins whenever

F · r < p        // 1024 × 0.1 = 102 tokens

p < 102      pay full price, the floor is irrelevant
102 .. 1024  below the floor, but padding to it strictly wins
p >= 1024    already caching

That middle band is where essentially every production system prompt we have ever written lives. A 900-token prompt is paying nearly nine times what it would pay padded. Which raises the only question that matters: does the filler make the model worse?

The experiment

Prior work is not reassuring about irrelevant context. Models get distracted by irrelevant information, they lose material in the middle of long contexts, and retrieval noise degrades answers. So we ran it rather than assuming.

500 deterministic, exact-match tasks in four groups: arithmetic, sorting, word extraction, and reading comprehension. The last group is the one that matters, because it forces the model to attend to a supplied passage, which is where filler could plausibly do damage. Three system prompts, same instruction core, three lengths:

condition	system tokens	caches	accuracy (n=500)	reading only (n=130)	paired diff vs clean
clean, below floor	38	no	81.8%	86.9%	baseline
junk padding	1,082	yes	82.0%	86.9%	+0.2 pp [-1.1, +1.5]
worked examples	1,032	yes	86.0%	87.7%	+4.2 pp [+1.8, +6.6]

The junk is a thousand tokens of lighthouse history, deliberately seeded with its own numbers and dollar amounts so that a distractible model would have something to grab. It changed nothing. The aggregate moved 0.2 points, which is noise, but the aggregate is the weak way to read this. Paired, 489 of 500 tasks came out identical under both prompts, 5 flipped toward clean and 6 toward junk, McNemar χ² = 0.0.

Report the bound, not the p-value

A large p-value is not evidence of no effect, and a null result you cannot bound is not a result. So the number we actually ship is the interval on the paired difference: any accuracy cost above 1.1 points is excluded at 95% confidence. The reading group is tighter still, 113 of 130 identical, exactly one item flipping in each direction, paired difference 0.0 points [-2.1, +2.1].

The third arm is the interesting one operationally. Pad with worked examples instead of junk and accuracy goes up 4.2 points, p=0.001, concentrated almost entirely in the format-sensitive extraction group which rose from 39% to 56%. That is ordinary few-shot prompting doing ordinary few-shot work. It is not evidence that length helps, and we are careful to say so, because that arm confounds length with examples. It does mean the useful version of this trick pays twice: the examples earn their keep, and cacheability arrives free.

The rule we shipped

if (tokens(prefix) < FLOOR) {
  // never with filler. examples, schema notes, edge cases,
  // anything a reviewer would defend in a code review.
  prefix = prefix + examples(FLOOR - tokens(prefix));
}
assert(tokens(prefix) % 128 === 0);   // do not donate the remainder

The assertion at the end is not decoration. The un-cached remainder averages 57 to 69 tokens across our prompts and peaks at 125, up to about 6% of a 2,000-token prefix, billed at full price on every call for the lifetime of the prompt. Landing on a block boundary is free money that nobody collects.

Scope, honestly: four short machine-gradable task groups, one provider, one model. We did not show that padding is universally safe, and the literature above is a real warning for long-context work. We showed that escaping this particular floor was free on our workload, which is enough to make the floor an accounting quirk rather than a law of nature.