Inference cost
The dollar cost of running an LLM to generate a response — calculated in input and output tokens, varying by model and provider.
Definition
Inference cost is the price paid each time an LLM generates a response. It breaks down into two components: the cost of input tokens (everything sent to the model: instructions, context, code) and the cost of output tokens (the generated response). Both are billed separately, with output generally 3-5x more expensive than input to offset the computational cost of generation. In an agentic coding workflow, input dominates: loading large code files costs more than the modifications produced.
postcursors perspective
Inference cost is a technical decision variable, not just a budget line item. Choosing between Claude Sonnet ($3/$15 per M tokens) and DeepSeek V3 ($0.27/$1.10 per M tokens) for a boilerplate test generation task isn't "being cheap" — it's engineering. The quality difference on that type of task is marginal, the cost difference is a factor of 10. Across a team of 5 devs with 30-50 sessions/day, that's $200-500/month difference. Multi-provider tools (OpenCode, Kilo Code) let you route intelligently based on task nature.
In practice
Concrete strategy: use a budget model (DeepSeek V3, Gemini Flash) for repetitive mechanical tasks (boilerplate tests, migrations, documentation, mass renames) and a premium model (Claude Sonnet, GPT-5) for tasks requiring deep reasoning (architecture, complex debugging, refactoring with business context). Routing by task type, not by entire session, is the most effective way to optimize.
Common misconceptions
- ✗ Optimizing only the output (cheaper model) without reducing the input (overly broad context) — that's where the real cost lies
- ✗ Comparing prices without looking at the available context window — a cheaper model with 32k tokens can cost more than a premium model with 200k tokens if the task requires long context