9 entries
A local proxy (RTK) shows 4.4M "saved" tokens (45.3%) across 9,892 cumulative commands (global scope). That figure reads like a lower bill; that reading is incomplete. That "saving" only exists because a harness (hooks, gates, routing) produces it and keeps producing it. A deliberate share of my AI budget has to fund that upkeep, not just more generation tokens.
A counter built into the tool can't measure the tool's savings: it sees neither the counterfactual, nor the cache, nor retries. Measured against my invoice, the real gain of four compression tools lands between 0 and 15%, while cache reads, meanwhile, make up 97% of tokens and re-read every token ~36 times.
By mid-2026, the reference LLM leaderboards have frozen or pivoted, and what stays current is either self-reported by vendors or recopied by unverified aggregators. One independent harness still holds — and it saturates at the top. Result: the source of truth left is your own codebase. Here's how I decide without leaning on numbers nobody can verify anymore.
Under a budget-capped subscription like opencode Go, picking a model isn't about token price but requests per 5 hours, the one quality source still alive (vals.ai), and tool-calling reliability — the decisive criterion no benchmark measures. The result: one model per preset, a frontier delegated outside the agent, and tools constrained rather than instructed.
Route mechanical tasks to budget or free models (MiniMax 2.5 via Kilo Gateway, Gemini Flash), reserve premium models for reasoning. Result: mechanical tasks (~65% of volume) cost nothing, with no perceived quality loss.
On mechanical, repetitive PHP/TypeScript tasks, budget models (MiniMax 2.5 via Kilo Gateway, Gemini Flash) produce results indistinguishable from Claude Sonnet. On complex reasoning tasks, Claude maintains a real edge. The takeaway isn't "use the cheapest" — it's "use the right model for each type of task."