benchmarks

2 entries

Saving tokens: the counter says 45%, the invoice 2%

A counter built into the tool can't measure the tool's savings: it sees neither the counterfactual, nor the cache, nor retries. Measured against my invoice, the real gain of four compression tools lands between 0 and 15%, while cache reads, meanwhile, make up 97% of tokens and re-read every token ~36 times.

tokens inference-cost claude-code

08/12/2026

Dispatch

Choosing a model when benchmarks can't keep up

By mid-2026, the reference LLM leaderboards have frozen or pivoted, and what stays current is either self-reported by vendors or recopied by unverified aggregators. One independent harness still holds — and it saturates at the top. Result: the source of truth left is your own codebase. Here's how I decide without leaning on numbers nobody can verify anymore.

model-routing benchmarks inference-cost

07/27/2026