In June I wrote about Headroom, a small free tool that shrinks the context an AI reads before your request goes out. I had been running it a week. My savings were in the single digits, and I had an explanation ready: my sessions are conversational, writing and thinking rather than churning through files and logs, which is the work the tool compresses least. Point it at heavy code work, I said, and the number will climb. I should be honest that I never really ran that experiment. I built two short games for my kids, and the rest of the last six weeks looks like the six before it. Text, mostly. What I did instead was leave the tool on and let it accumulate, and it has now handled 3.46 billion tokens. The number is still in the single digits. It is 3.8%.
Six weeks, in numbers
So I went looking for the reason, expecting to find my own workload staring back at me. It was not my workload.
The whole ledger, straight off my machine.
The last eight days on their own came to 28.5 million tokens saved, around 3.5 million a day, running at 3.2%. That is a little under the lifetime average. Eight days is not enough to call it a trend, so I am noting it rather than reading anything into it.
One thing to be straight about before going further. The tool puts my savings at $734.99 against a total input cost of $13,757. I never paid $13,757. I am on a subscription, so that figure is what the same traffic would have cost at pay-per-call rates. Useful as a yardstick, useless as a receipt. What compression actually buys me is room under the weekly cap. The tool is called Headroom, and that turns out to be the honest description of what it hands you.
The thing that limits it
The answer showed up in a request-by-request breakdown of one working session. 971 requests. 622 got compressed, averaging 4.7%, the best single one cut by a quarter, from 51,083 tokens down to 38,075. That part does what it says.
The interesting column is the other one. 312 requests, close to a third of the session, were skipped and labelled "prefix-frozen." Another 32 had nothing compressible in them at all.
Prefix-frozen means the provider has already cached that content. Worth a short detour if you have not run into this. When you send the same block of text to a model over and over, which is exactly what a long conversation does, the provider can store it and charge you a fraction of the usual price to read it again. With Anthropic that fraction is about a tenth. The catch is that the cache only works if the text is identical every time, byte for byte. Change something near the front and the whole thing gets paid for again at full price.
Which means compression and caching want opposite things. Compression rewrites your context to make it smaller. Caching pays you for never touching it. Shrink a block that was already cached and you trade a small discount for a large one, in the wrong direction.
Headroom appears to know this and steps back. A third of my traffic gets deliberately left alone, because compressing it would cost me more than it saves. That is not the tool failing. That is the tool being right, and a good share of my missing savings is the price of it being right.
When it does fire, it fires hard
The rest of the picture makes the low average look different again. Across 1,029 distinct content patterns and 6,512 compressions, the tool examined 5.09 million individual items and kept 3.28 million. It dropped 1.8 million, about 35% of everything it looked at. Weighted across all of it, when compression actually runs it removes 38.3% of the tokens.
So the tool is not weak. It is picky. My 3.8% is a blend of roughly 38% on a thin slice of traffic and nothing at all on a thick one. Those are different problems, and only one of them is worth complaining about.
One more number I did not expect. Against those 6,512 compressions, the tool logged 5,100 retrievals, which is the model asking for the original of something that got squeezed. Roughly four pull-backs for every five compressions. I had filed that feature under safety net. It is closer to routine.
For calibration: the default profile ships targeting a 90% reduction. I never changed it, and did not know until this week that it could be changed. Reality lands at 3.8%. That target is a ceiling on how hard the tool squeezes what it chooses to squeeze, not a forecast of your bill.
I am not the only one measuring this
While I was digging I found someone who had done the harder version of this test. They replayed a 614-million-token corpus of their own past Claude Code sessions, carrying $926 of real spend, through Headroom and two similar tools.
Headroom alone saved 2.8% of that spend. All three together saved 3.7%. Their conclusion is worth quoting in spirit: none of the advertised numbers are exaggerated. On the content these tools are built for, the cuts run from a third to nearly everything. It is the full session that dilutes them, because most of a session is not that content.
My 5.3% of notional cost is roughly double their 2.8%, on a corpus about five times larger. I do not think either of us is wrong. Two people measuring their own traffic got two single-digit numbers, and both sit a long way from the 60% to 95% on the box.
There is a real counterargument to treating small numbers as unimportant, and it comes from the same review. A tool result does not get paid for once. It gets re-sent on every later turn of the session, so compressing it once saves you repeatedly. In their measurement, 1.5 million first-turn tokens became 41.3 million across the full sessions, about 27 times as much. That is the same compounding I wrote about in June, arriving from someone else's data.
About that cache column
Here is where I want to be careful, because there is a genuine argument happening online about it and I do not think my data settles it.
The live session summary reports two columns side by side: about $20 saved by compression, and about $1,587 attributed to caching. Eighty to one. It would be easy to read that as the tool taking credit for a discount the provider was already giving me, since Claude Code caches whether or not Headroom is installed.
Two things complicate that reading. Headroom's own persistent ledger, the file it accumulates over months, records only the compression figure. No cache line at all. It is not banking the big number as its own. And part of what the tool actually does is protect that cache, keeping timestamps, session identifiers and other shifting content out of the stable prefix so the discount survives.
What nobody has established, including me, is the counterfactual. I do not know what my cache savings would look like without it. Anyone quoting that 80-to-1 gap in either direction, as proof the tool is redundant or proof it is doing something huge, is going further than the data goes.
The bigger point
I have been writing about efficiency as though it were a pile of separate wins you collect and add up. Cheaper model, smaller context, cache what repeats. It does not add up like that. Compression and caching pull against each other, and the useful work sits in deciding which one a given request deserves.
My best six weeks came from a tool that spent a third of its time choosing not to do the thing it exists to do. I keep turning that over. The number on the box was never the interesting part.
Headroom is free and open source, built by chopratejas, at github.com/chopratejas/headroom. Every figure above is from my own install, version 0.27.0, over six weeks of daily use.