The AI FinOps levers that actually move your LLM bill

Since the beginning of the GenAI wave, the industry has optimized for one thing: usage. Ship fast, scale blind, and let someone else worry about the bill. Well, the bill has arrived. As AI features have moved from demos to production and adoption has taken off, token spend has become a real line item, and teams are discovering that their AI costs sometimes grow faster than the value they deliver. And because most providers are still selling inference at a loss, the pressure will only intensify as they head toward IPOs and can no longer afford to subsidize your workloads. In short: cost is no longer a future concern.

This talk is a practical tour of the cost-reduction toolbox. The rule-based techniques (e.g. rtk) that worked well for coding agents don't transfer to most other applications, but there are other levers you can act on. Some are more accessible, like prompt caching optimization (we'll see how providers implement it differently, and why you should track cache behavior rather than trust the process), or rethinking model choice through automated benchmarking and semantic routing: a smaller model may perform just as well on a specific task, and the new generation of open-weight models (mostly from China), designed for inference efficiency, now delivers strong performance at a fraction of the cost. Others demand more investment, like fine-tuning for repetitive workloads.
Finally, we'll explore the future of optimization with emerging techniques like token eviction, which lets you feed the model only what it needs without breaking your caches.

You'll leave with a mental map of the AI cost landscape and concrete levers to pull.