For the first time since the generative AI boom started, enterprises are spending more money running models than building them.
Gartner's 2026 forecast puts worldwide spending on AI-optimised infrastructure at $42 billion this year, and for the first time, more of that money goes to inference than to training: $23.3 billion against $19 billion. It is a quiet but structural shift, and most finance teams have not caught up with it yet. The FinOps Foundation's State of FinOps 2026 report, covering nearly 1,200 organisations and $83 billion in annual cloud spend, found that AI now makes up 18% of cloud spend at AI-forward enterprises, up from just 4% in 2023. The same report found that 73% of those organisations saw their AI costs exceed the original budget. The average enterprise AI budget has grown from $1.2 million a year in 2024 to $7.0 million in 2026, and 80 to 90% of that spend now sits in inference rather than training. Training is a job that finishes. Inference starts the moment you ship and keeps running for as long as anyone uses what you built.
We are seeing this pattern directly in client conversations. Teams that built an agent, proved it worked, and rolled it out to production are the ones now asking a different question: not "does this work" but "what does this actually cost us, and is it worth it." That question deserves a proper answer, not a shrug.
Part of the problem is architectural. Agentic workflows chain several model calls together to complete one task, and every step in that chain adds tokens, adds latency, and adds cost. A single request that used to be one API call to a chatbot can now be five or six calls inside an agent's reasoning loop, retrieval step, and tool invocation, each billed separately.
The other part of the problem is failure mode, and it is a more expensive one than most teams plan for. Analytics Week's 2026 analysis on inference economics documents recursive agent loops, where an agent gets stuck retrying or re-reasoning without a proper exit condition, generating roughly $3,700 in unplanned compute over three hours for a single agent, and around $37,000 for the same failure across ten agents running simultaneously. Across the industry, Analytics Week estimates this pattern alone absorbs close to $400 million a year. None of that spend produced anything. It is pure waste, and it is largely preventable with the kind of guardrails and escalation logic we have written about separately for agent governance.
There is also a quieter form of waste sitting underneath the dramatic failures. GPU utilisation during live inference workloads commonly runs at only 15 to 30%, according to the FinOps Foundation's data, which means most organisations are paying for capacity they are not using most of the time.
Cost control for inference is not one lever. It works in layers, and the order matters, because decisions made earlier in the chain affect every dollar spent afterwards.
Start at the model layer. Not every task needs your most capable, most expensive model. Routing simple classification or extraction tasks to a smaller model and reserving the frontier model for genuinely complex reasoning is the single highest-leverage decision most teams can make, and it is also the one most commonly skipped, because in early development it is simplest to send everything through one model and worry about efficiency later.
Then look at the runtime. Quantisation, where a model's weights are compressed to a lower precision, can cut GPU memory requirements substantially and roughly double or triple throughput, with minimal quality loss for common enterprise tasks like classification and document analysis. This is a well-understood technique, not an experimental one, and it is usually the first thing worth trying once a model is confirmed to work.
Infrastructure choice comes next. Reserved instances and savings plans deliver savings of 40 to 72% over on-demand pricing for workloads that run predictably, and spot instances can save 60 to 80% for work that tolerates interruption, such as batch inference or offline report generation. Neither is appropriate for real-time inference under a strict service level agreement, which is exactly why this decision has to be made per workload rather than applied as a blanket policy.
FinOps runs across all of it, continuously, not as a one-off audit. That is the discipline itself: token budgets by team or product line, cost attribution down to the model and the workflow that consumed it, and someone whose job it is to notice when a number moves before it becomes a line item nobody can explain in a board meeting.
The harder problem is not measuring cost. It is connecting that cost to something the business actually cares about. Forrester's most recent survey of AI decision-makers found that only 13% of organisations report a positive EBITDA impact from their AI investment, and fewer than a third can link AI's contribution directly to profit and loss. Fewer than one in three decision-makers can currently tie AI's value to their organisation's financial growth at all. Forrester expects enterprises to defer roughly a quarter of planned AI spend into 2027 as a direct result of this measurement gap, not because the technology stopped working.
This is usually not a technology problem. It is what happens when AI is treated purely as a cost centre, run by a technology team optimising for efficiency, with no clear owner translating "the agent processes tickets faster" into a number a CFO can put in a forecast. The fix is to define the outcome before the workflow ships, not after. If an agent is meant to reduce time-to-resolution on support tickets, decide upfront how that maps to cost per ticket, and track both the inference spend and the resolution metric side by side from day one. If the two numbers cannot be pulled up in the same conversation, the initiative is not actually being measured, whatever the dashboard says.
Software delivery has its own version of this measurement problem, and it cuts both ways. Google Cloud's DORA research for 2026 found that AI adoption was associated with a 7.2% reduction in software delivery stability for every 25% increase in adoption, a reminder that faster output and healthier delivery are not automatically the same thing. Separate analysis using GitClear data found copy-pasted code climbing from 9.4% of new code in 2022 to 15.7% in early 2026, with refactored code falling sharply over the same period, both signals of rework accumulating quietly behind a productivity number that looks good on its own. None of this means AI coding tools are a bad investment. A genuine 10% productivity gain still comfortably justifies a typical $19 to $39 per-developer monthly licence cost. It means the licence fee was never the real cost to track. The real cost sits in code review load, rework, and delivery stability, and those only show up if someone is measuring them alongside the AI spend, not instead of it.
The instinct to reach for the largest, most capable model for every task is understandable, and increasingly the wrong default.
A growing body of evidence points the other way. LinkedIn's EON-8B model, built on a much smaller open-weight base and fine-tuned specifically for candidate-job matching, matched or beat larger general models like GPT-4o on that task while costing meaningfully less to run. A 2026 study on contract clause extraction found that a domain-trained small language model outperformed frontier general-purpose models on the task while operating at a fraction of the inference cost.
Gartner's framing for this is domain-specific language models, models built or tuned for a particular industry, function, or class of problem. Their view is that the next phase of enterprise AI will not be decided by which vendor has the largest general model, but by who can combine domain expertise, governance, and deployment flexibility into one coherent system. That maps closely to what we see in practice: a hybrid architecture, where a large general-purpose model handles broad reasoning and ambiguous requests, and smaller, cheaper, purpose-built models handle the narrow, high-volume, repeatable work sitting underneath it, tends to outperform a single large model doing everything, on both cost and reliability.
The practical version of this is a routing decision, made deliberately rather than left to whichever model was easiest to wire up first. Not every ticket needs a frontier model's reasoning. Most need a fast, accurate, narrow response, and a smaller model tuned to that specific job will usually deliver it for a fraction of the cost, with a general model held in reserve for the genuinely hard cases.
The token price per unit of intelligence has fallen sharply over the past two years, and total enterprise AI spend has still gone up. That is the paradox sitting underneath most AI budgets right now: getting cheaper per call, growing faster in total, because usage, not price, is what is actually compounding. Scaling an agentic system sustainably means treating that growth as something to architect for, not something to discover after the invoice arrives.
In practice, that means: know which workloads justify a frontier model and which do not, build the routing and caching logic that keeps token consumption proportional to the value produced, put escalation and loop-detection guardrails in place before an agent runs unattended at scale, and give someone in finance and someone in engineering shared visibility into cost per outcome, not just cost per token. Organisations that build this discipline in early are not the ones moving cautiously. They are the ones able to scale with confidence, because they can see what scaling actually costs before they commit to it.
You do not need a full FinOps function to start closing this gap.
You need one workflow where inference cost, business outcome, and model choice are all visible in the same view, built as the template for the next one. That is a smaller project than it sounds, and it tends to surface the routing and architecture decisions that matter most within the first few weeks.
We have spent over a decade helping clients turn software plans into systems that hold up under real operational and financial scrutiny, and the AI economics conversation is following the same pattern we have seen before: the technology gets cheaper and faster every year, and the organisations that win are the ones disciplined enough to architect for that, rather than discover it in a budget review.
Let's talk about how this framework applies to your product.
.jpeg)
Because usage is compounding faster than price is dropping. Per-token pricing has fallen sharply over the past two years, but total enterprise AI spend has grown from roughly $1.2 million a year in 2024 to $7.0 million in 2026, largely because agentic workflows chain multiple model calls together and inference, unlike training, runs continuously once a system ships.
Model routing. Sending simple, well-defined tasks to a smaller, cheaper model and reserving frontier models for genuinely complex reasoning is the highest-leverage decision most teams can make, and it is usually the one skipped early on because it is simpler to route everything through one model.
Track the business outcome and the inference spend side by side from day one, not as two separate reports. If an agent is meant to cut ticket resolution time, define upfront how that maps to cost per ticket. Most organisations struggle with AI ROI not because the technology underperforms, but because nobody defined what to measure before the workflow went live.
