The Hidden Costs of AI at Scale

On June 1, 2026, GitHub permanently retired flat-rate “premium requests” for Copilot and replaced them with usage-based AI credits. When the first invoices under the new model arrived a month later, some agentic users saw bills they were not prepared for: One developer reported monthly costs jumping from $29 to $750 for the heaviest agentic workflows.

It was one visible example of a broader shift across the AI tooling market over the course of 2026 – and one that may await organizations still paying a flat rate today.

Organizations count the hours they save. Many of them do not count what a flat rate keeps invisible: context consumption and retries after failure. Other costs never appear on the vendor invoice at all, including the time spent reviewing output and maintaining prompts. Once billing shifts to actual consumption, organizations without cost discipline risk a bill that surprises them in much the same way Copilot’s new model surprised some users.

The Context No One Priced In

AI obviously needs context; that’s not in question. The question is whether the context being sent is relevant, or just conveniently available. Sending an entire document is the fastest way to give a model information. It isn’t automatically the cheapest or the best way.

In May 2026, Stanford’s Digital Economy Lab published an analysis of agentic coding tasks across eight frontier models and found that these tasks consume up to a thousand times more tokens than a simple code chat, with the main driver not being the model’s output but the input context it re-sends over and over. The agent re-reads its entire history at every subsequent step. The same task, run multiple times, varied in token consumption up to thirtyfold.

Accuracy doesn’t scale linearly with context volume either: it often peaks at a moderate amount and then just adds cost without adding value.

Token blindness, then, isn’t about AI not needing context. It’s about the fact that without measurement, no one asks whether all that context is actually necessary. Under a flat rate, the question is easy to ignore. Under consumption-based billing, it becomes part of the cost.

When You Pay for Failure Twice

Agentic workflows carry another cost that almost never shows up in ROI calculations. Picture a simplified chain of ten steps, each with a 95 percent chance, on its own, of running correctly. That sounds reliable enough, but strung together, that chain has only about a 60 percent chance of making it through the whole run without a single error.

In a workflow that sends the accumulated context again on every call, each failure and the retry that follows doesn’t just cost you the repeated step: you pay again for everything sent before it too.

It’s a common pain almost everyone goes through when building their first agentic pipeline. I went through it myself. Early on, with just a couple of agents, it didn’t matter much. But as the pipeline grew, every failed run got more expensive, and that’s what pushed me to start asking what context each agent needed and how to cache it, instead of just whether the run went through.

The same analysis calculates that a ten-step agent with 95 percent reliability per step burns roughly 40 percent more tokens on retries than a perfectly reliable system. This is a cost you’ll see on the invoice but, likely, won’t be found in any ROI spreadsheet.

Oversight Isn’t a Bug. It Belongs in the Budget

This point needs to be made precisely, because it’s easy to get wrong. Reviewing AI output isn’t a system failure; it’s a legitimate, expected part of working with AI, the same way code review is a legitimate part of working with developers. The problem isn’t that output gets reviewed. The problem is that this work almost never gets factored into the calculation of how much AI actually saved.

Glean’s Work AI Institute surveyed 6,000 workers and found that automation saves them about 11 hours a week, but nearly six and a half of those hours go into maintenance tasks: giving AI systems context, checking their work, and cleaning up mistakes. The net saving, then, is closer to four and a half hours – less than half the headline figure. AI still saves time, just not as much as the first number suggests.

Prompts Need Maintenance, Not Just an Author

Prompts behave more like production code today: a model update, a change in context or seemingly minor edit can alter how they perform. Without versioning and testing, those changes can quietly introduce problems. The regression tests that are standard practice for code are still often skipped when it comes to verifying prompts. A change that looks like a minor edit to a single sentence can reach production and reduce accuracy without anyone noticing until the problem has accumulated into something visible.

Building a proper evaluation framework – including a test set and automated regression tests on every change – is extra work that almost never shows up in the “AI saves time” calculation.

Cheaper Tokens, Higher Bills

GitHub Copilot wasn’t an exception. A survey cited by CFO Dive found that nearly seven in ten U.S. companies reported at least partial AI budget overruns over the past year, mostly before a full shift to consumption-based billing, not after it.

Bain & Company, in its June analysis of token economics, adds a paradox that captures the whole situation best: the price per token fell by half over the year, while consumption over the same period grew 4.5 times.

The model got cheaper, but the bill stays stubbornly high. Companies moved to newer models, gave agents more complex tasks and found more workflows for them. A cheaper token did not mean lower spending; it meant there were more reasons to consume one.

How To Prepare Before the Bill Arrives

The framework that follows from this isn’t about using less AI. It’s about knowing what AI costs before you decide to scale it further.

1. Get visibility first

Until you have consumption broken down by team, workflow, application and completed task, every expansion is a blind bet. That visibility isn’t free either: for agentic workflows especially, tracing each step, logging what happened and why, and monitoring for runaway loops takes its own engineering time and tooling. Budget for it as part of the cost of running AI, not as an afterthought on top of it.

2. Recalculate ROI on a net basis

Subtract the time spent on review, fixes and prompt maintenance from the reported hours saved. If time saving is the purpose of the use case and the net result is negative or unverifiable, it is not ready to scale. Where the intended benefit is quality, capacity, risk reduction or revenue instead, measure that outcome directly.

3. Apply cost discipline, but not uniformly

A hard spending cap makes sense where failure is cheap: internal tools, experimental agents, dev environments. For critical, customer-facing functionality – a customer-service assistant, for example – a hard cap isn’t workable, because it creates an outage risk. There, you need tiered fallbacks to a cheaper model and early-warning alerts, not a shutoff at zero.

4. Treat prompts and evals as engineering assets

Version them, test them and review changes before deployment, the same way you would manage production code.

Walk into renewals with your own data

Vendor pricing is difficult to evaluate without your own usage data. Before a renewal or model change, calculate what your existing workflows would cost under the proposed terms. The goal isn’t only to negotiate a lower price. It’s to know how that price will behave at your actual level of consumption, instead of finding out from the invoice.

Three things you can do this week: check whether you can break down AI consumption by team and workflow; choose one use case and put the time spent on review next to the hours reportedly saved; and find out where a hard spending cap could create an outage instead of controlling a cost.

AI costs can be managed. Just not when you are finding out what they are for the first time from the bill.