For most of the software era, the cost of serving one more customer was close to nothing. You wrote the product once, and the marginal cost of the next signup was a little storage and a little bandwidth. That is the reason SaaS businesses have carried gross margins above 80%. The arithmetic held for a long time.
Products built on large models do not work that way. When your product calls a model, every request costs money, and the cost rises with how much text goes in and comes back. The bill is no longer a fixed platform expense. It is a variable input that scales with usage, which is the textbook definition of a cost of goods sold.
Why agents changed the shape of the bill
A single question to a model is cheap. The change came when the unit of work grew. Agents now run for many steps to finish a task. Coding tools loop over a repository. Assistants hold conversations that carry on for hours. The work is a session, not a message, and a session sends far more tokens than a one-off answer ever did.
The mechanism is worth being precise about. At each step, the model has no memory of the last one, so the whole context goes back in: the system prompt, the running history, the tool definitions, the documents that were retrieved. A task that takes forty steps pays for that context forty times, and the context grows as the task goes on. Research on long agent sessions finds that as much as 90% of what gets sent can be context that was already sent earlier in the same session. You are paying, over and over, to resend text the model has effectively already seen.
What the numbers look like
This is not a rounding error on the income statement. Enterprise spend on model APIs reached $8.4B in 2025, up 2.4 times in a single year (Menlo Ventures). Industry analysts expect the inference market to grow into roughly $105B by 2030. At the company level the pressure shows up in margin: AI-native software is running at 30 to 50% gross margin, against the 80%-plus that traditional SaaS has enjoyed (ICONIQ and public reporting, 2026). At one category leader, inference runs $0.40 to $0.70 of every revenue dollar (public reporting, 2026).
When a cost grows faster than the revenue it supports, it sets the ceiling on the business. A team can win more customers and still watch margin slip, because each new customer brings more sessions, and each session brings more resent context. The pricing page holds steady while the cost underneath it climbs.
What a company can do about it
The useful part is that a cost of goods can be managed like one. Manufacturing businesses have spent a century learning to hold a unit cost inside a target while keeping the product good enough to sell. The same discipline applies here, and it starts with looking at the whole session rather than the single call.
Looking across the whole session, rather than one call at a time, is what makes the bill manageable. The hardest step and the most routine step do not need the same model, and much of a session is context the model has already seen, so a session need not be paid for at full rate every time. Two controls turn that into something you can rely on: a budget, a ceiling on what a session or a day is allowed to cost, fixed before the work runs; and a quality floor, a standard the routing is not allowed to drop below, so cost never comes at the expense of the answer. Set both, route the whole session inside them, and the bill stops being a number you discover and becomes one you decide.
That is the work we do at Moji Router, the management layer for AI inference: most of the bill is context sent more than once, and routing the whole session inside a budget and a quality floor is how we bring it down.