Fellowship Intelligence
The Wrong Default

Vendor & counterparty risk

You Can Price the Invoice. You Cannot Price the Work.

Nobody at the vendor set your number. Nobody at your company can predict it.

An arcade card converts your money into a currency the arcade invented. Every game costs a different number of those units, the exchange rate is posted by the door, and when you leave you are never quite sure what the evening cost per game. Nobody minds, because nobody walks into an arcade believing they are doing procurement.

The problem with AI billing is not that it behaves like an arcade. It is that it behaves like an arcade while being presented as enterprise pricing, and signed by people who believe they bought a unit price.

Four vendors, four invented currencies

Look at how the largest enterprise platforms meter their AI today.

Microsoft moved Copilot Studio from “messages” to Copilot Credits on September 1, 2025. A prepaid pack is 25,000 credits for $200 a month. A generative answer costs 2 credits, an agent action 5, tenant graph grounding 10. Microsoft’s documentation says unused credits “don’t carry over to the next month,” and that if usage exceeds purchased capacity, “technical enforcement applies and can result in service denial.”

That is the arcade card, almost exactly. You buy credits at the door. Each game costs a posted number of them. When the card runs out, the games can stop. And whatever is left on the card expires at the end of the month.

ServiceNow meters assists. Its legal overview, effective August 24, 2026, defines “1 Action = 1 Tool Invocation” and prices agentic workflows in tiers: 25 assists for a small workflow, 50 for 5 to 8 actions, 150 for 9 to 20. Anything past 20 “will be counted in a new agentic workflow.” So a workflow that makes eight tool calls costs 50 assists, and one that makes nine costs 150. One extra tool call triples the price. And who decides whether the agent makes that ninth call? Not the buyer. The agent does.

Picture a racing game that decides, halfway through the race, to add a lap, and then charges you triple for the longer course.

The same document charges 200 assists per month for turning on the managed flag on an AI asset in AI Control Tower. Governance is metered on the same card as the work. The more carefully you watch your AI, the more the watching costs.

Salesforce sells Flex Credits: $500 buys 100,000, and one Agentforce action consumes 20 of them, or $0.10. It also still lists a flat $2 per conversation. Those two prices cost the same only when a conversation takes exactly 20 actions. Fewer, and paying per action wins. More, and paying per conversation wins. Whichever model the buyer is on, the agent, by deciding how many actions to take, decides whether it was the better deal.

Cognition bills Devin’s enterprise customers in ACUs, which “represent the work performed by Devin.” The rate per ACU is negotiated and “set in their order form,” which is exactly how enterprise procurement is supposed to work. But the number of ACUs a job consumes is not in the order form. It accrues with the “number and complexity of actions Devin takes,” along with machine time and bandwidth. You negotiated the price of the unit. Devin decides how many units the work takes, and when the limit is reached, “new work billed to it is blocked.”

Credits, assists, Flex Credits, ACUs. Not one of them is a token, and not one of them converts cleanly into another. Two quotes from two vendors for the same job are denominated in two currencies that neither vendor will exchange for you.

Underneath the credits, the token

Every one of those invented currencies sits on top of another one: the token. And the token is controlled by four variables, none of which belong to the buyer.

1. What a token is. Tokenization is a vendor implementation detail. Anthropic’s pricing page states it plainly: its newer models “use a newer tokenizer” that “produces approximately 30% more tokens for the same text.” Same vendor, same words, a different count after an upgrade.

We checked this ourselves against OpenAI’s two public tokenizers. An English sentence came out the same in both, 11 tokens. The same sentence in Spanish was 16 tokens under the older encoding and 13 under the newer. In Japanese it was 29 and 20. Nothing about the sentence changed. The meter did.

2. What a token costs. A token is not one price. Anthropic publishes five for Claude Opus 5.5: $4 per million input tokens, $5 or $8 to write to the cache depending on how long it is kept, $0.20 to read from it, and $20 for output. OpenAI’s gpt-6-astra charges $10 per million input tokens, $1 for cached input and $50 for output. Google’s Gemini 3.1 Pro Preview charges $2 and $12, and counts its thinking tokens as output.

Even the shape of the rate card varies. OpenAI doubles the input rate on prompts over 272,000 tokens, applied to the full request. Google raises its rates above 200,000. Anthropic dropped its long-context premium entirely. Three vendors, three different answers to the same question, and any of them can change on the next pricing update. The structure of the rate card is a choice, and it is not yours.

3. What tools get called. The harness decides, and tools cost money. Tool definitions are billed as input tokens at both OpenAI and Anthropic, and Anthropic adds a tool-use system prompt of its own on top. Web search runs $10 per 1,000 searches at both, plus the tokens the results bring back. Google bills “for each search query that the model decides to execute.” That is the vendor’s phrasing, and it is the whole point: the model decides.

4. How long the reasoning runs. This is the line item most likely to surprise a finance reader. At all three providers, reasoning tokens are billed as output. OpenAI’s documentation says they “are not visible via the API” and that “you could incur costs for input and reasoning tokens without receiving a visible response.” Anthropic’s says “the billed output token count does not match the count of tokens you see in the response.” Google’s says that if a response hits its limit, it can come back “truncated or empty” while “still billing for any thinking tokens generated.”

And the amount of reasoning is not set by you. At Anthropic the model “evaluates each request and decides for itself whether to think and how much,” and the effort setting “doesn’t guarantee a token count.” At OpenAI the models “reason adaptively.” Google uses “dynamic thinking by default.”

Then there are my favorites: the defaults. On Claude Opus 5.5, adaptive thinking “is always on and can’t be turned off,” the default effort is now medium where earlier Opus models defaulted to high, and the thinking text is not shown unless you opt in. Google’s Gemini 3.1 Pro, meanwhile, defaults to thinking on high. None of this is hidden; it is all documented. But if you did not choose those settings, the vendor chose them for you. If you upgraded, both what you get and how much of it you pay for may have changed, and nobody sent you a new price sheet.

Nobody set the price

Here is the part that makes this a governance problem rather than a pricing complaint.

Nobody at the vendor decides your invoice. There is no pricing manager setting your number. People designed the rate cards, the tokenizers, the harnesses and the defaults. But the number itself emerges from a process, per run, and it emerges differently every time.

That is an unowned decision. Nobody at the vendor set it. Nobody at the buyer can predict it. And nobody at the buyer can audit it, because the reasoning tokens on the bill are not the tokens in the response. The policies are documented. The count is not verifiable. And the usual procurement lever, calling your rep and negotiating the rate, does not reach the mechanism, because the rate was never the variable. The quantity was.

Batch discounts (50% at all three providers) and prompt caching are real, and they help. They are also rate levers. They lower the price of a token. They still do not tell you how many tokens the job will take.

Same job, different bill

How much does the quantity move? Researchers from Michigan, Stanford and MIT ran eight models through all 500 tasks of the SWE-bench Verified coding benchmark, four times each. Their finding: “token usage is highly variable and inherently stochastic: runs on the same task can differ by up to 30× in total tokens.” The typical case is milder but still breaks a budget: on the same problem, the most expensive run cost about twice the cheapest. More tokens did not mean better answers. And when asked to predict their own token usage, the models could not do it, with correlations no better than 0.39.

Princeton’s Holistic Agent Leaderboard, which logged more than 21,000 agent runs, found that on one web-browsing benchmark one agent setup cost $171 and another cost $1,577 for a two-point difference in accuracy. A 9x spread, driven mostly by the harness around the model.

Even the “same run” is not the same. Thinking Machines Lab ran one prompt 1,000 times at temperature zero, the setting that is supposed to be deterministic, and got 80 different completions.

So after the fact, you can see exactly what one run cost. What you cannot say is what the job costs, because the next identical job will cost something else. Unit economics assumes the same input produces a comparable cost. Here the unit is vendor-defined and the quantity is model-determined, so the assumption fails at the root rather than at the margin.

We ran this ourselves, twice. First, a simple job: look up one vendor’s pricing and write a 250-word briefing. Five identical runs on OpenAI’s gpt-6-astra, default settings. The most expensive cost 28% more than the cheapest. Then a more realistic one: research three vendors’ agent pricing, build a comparison table and write a 400-word memo for a CFO. Six identical runs:

The memos we could read were nearly the same length every time. What we could not see was not. The model pulled in 59,600 tokens of material, or 91,600. It reasoned for 1,306 tokens, or 3,303. The most expensive run cost 56% more than the cheapest. Ten thousand of these a month is $7,321, or it is $11,400, and nobody can tell you in advance which.

We then checked OpenAI’s invoice against its own published rate card. It matched to the cent. You can price the invoice. We did. You cannot price the work.

“Lawyers bill by the hour, and nobody calls that unaccountable”

This is the common objection, so I will address it directly.

It is true that variable cost is not unique to AI. Cloud egress varies. Database queries vary. A legal matter takes as many hours as it takes, and the client does not control that.

But the unit is externally defined, stable across vendors, and independently verifiable. You can benchmark one firm’s hours against another’s because both are selling the same unit.

Here, both sides of the multiplication sit inside the vendor’s system. The unit is vendor-defined, differs between vendors, and changed at one of them by roughly 30% on a single model upgrade. The quantity is determined by a model whose behavior the vendor also controls, through defaults the buyer may never have looked at. An hourly bill is unpredictable. This bill is unpredictable and incomparable.

Put plainly: an hour is an hour from one law firm to the next. A token is not a token from one vendor to the next.

The counterexample proves the point

Some vendors do sell outcomes instead of tokens. Intercom prices its Fin agent at $0.99 per outcome, where an outcome is a confirmed resolution or a handoff, charged “once per conversation.” Zendesk bills AI agents only for “Verified Resolution,” and verifies it “through LLM verification.”

This matters, because it shows that per-token pricing is a design choice, not a technical necessity. A vendor can take on the variance and sell you a unit of work.

It is also worth noticing what moved. Under outcome pricing, the unknown shifts from “how many tokens” to “what counts as an outcome,” and at Zendesk, a model decides that too. Even the vendor’s accountants have to estimate it: Deloitte’s June 2026 guidance on outcome-based pricing notes that vendors may need to “estimate at inception the total variable consideration” of the contract. The variance does not disappear. Someone just has to own it. That is the question every buyer should be asking: in this contract, who owns the variance?

And when a model is deciding what gets billed, the question gets harder. Is there a person on either side of the contract who owns that number? Not by default. But someone can own what counts as an outcome, the right to audit how it was counted, and the budget it runs against. That is not a pricing problem. It is a governance assignment.

Why this is where scaling stalls

In our September 8 edition we argued, on Gartner’s own framing, that what separates organizations that scale AI from those that stall is whether they can prove what happened after the money went in. This is the mechanism underneath that. It is hard to prove return on a unit of work when you cannot say what the unit costs.

The warnings are not new. In October 2024 Gartner told CIOs they “could make a 500%-1,000% error in their cost calculations.” IDC wrote last December that large organizations will face up to a 30% rise in underestimated AI infrastructure costs by 2027, and that the reason “isn’t simply overspending.” It is “under-forecasting.”

The Decision Ownership Check

Before you sign or renew, name an owner for each of these:

1. The limits, not just the rate. You cannot control how many units a job consumes. Who owns the caps, the settings and the budget it runs against, and decides what happens when a job goes over?

2. The spread. Can you state what one completed unit of work cost, as an average across runs with a range, rather than as one invoice?

3. The defaults. Which vendor settings for reasoning effort, thinking, tools and context are you running on, and did anyone on your side choose them?

4. The currency. If two vendors quoted this job, could you convert both quotes into the same unit?

5. The watching. Is governance and observability metered on the same card as the work?

6. The alternative. Could this workload be bought per outcome, and what would the vendor count as one?

None of this makes the vendors the villains. Most of them publish more cost detail than software vendors ever have. The complaint is not that they hide the meter. It is that a fully disclosed meter on an unstable unit still does not give you a price.

Nobody set the price. So someone on your side has to own it.


Full disclosure, as promised yesterday: I am selling something. The Consigliere is a library of more than 100 written guides for using AI at work, organized by profession, industry and role. It has no AI in it and never sees what you type. You open the guide written for your work, use its prompts in the tool you already have, and treat what comes back as a draft to question. Four tools are free without an account, including Before You Adopt, a checklist for the moment before a new AI tool comes in. The library is open by invitation during evaluation at theaiconsigliere.com.


Sources (all accessed 2026-09-28 unless dated)

• Microsoft, Copilot Studio billing and licensing (updated 2026-08-03): https://learn.microsoft.com/en-us/microsoft-copilot-studio/billing-licensing

• Microsoft, Copilot Studio credit rates (updated 2026-08-03): https://learn.microsoft.com/en-us/microsoft-copilot-studio/requirements-messages-management

• Microsoft, Copilot Studio pricing: https://www.microsoft.com/en-us/microsoft-365-copilot/pricing/copilot-studio

• ServiceNow, Now Assist Overview (effective 2026-08-24): https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/legal/sn-assist-overview.pdf

• Salesforce Help, Agentforce pricing (2025-05-19): https://help.salesforce.com/s/articleView?id=004811240&language=en_US&type=1

• Devin docs, enterprise billing and usage:

https://docs.devin.ai/admin/billing/enterprise
https://docs.devin.ai/admin/billing/usage

• Anthropic, pricing: https://platform.claude.com/docs/en/about-claude/pricing

• Anthropic, thinking / adaptive thinking / effort: https://platform.claude.com/docs/en/build-with-claude/thinking https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking https://platform.claude.com/docs/en/build-with-claude/effort

• OpenAI, pricing and gpt-6-astra model page: https://developers.openai.com/api/docs/pricing https://developers.openai.com/api/docs/models/gpt-6-astra

• OpenAI, reasoning guide: https://developers.openai.com/api/docs/guides/reasoning

• OpenAI, function calling: https://developers.openai.com/api/docs/guides/function-calling

• Google, Gemini API pricing (updated 2026-09-24): https://ai.google.dev/gemini-api/docs/pricing

• Google, thinking (updated 2026-09-25): https://ai.google.dev/gemini-api/docs/thinking

• Google, grounding with Search: https://ai.google.dev/gemini-api/docs/google-search

• Tokenizer comparison: computed by The Wrong Default on 2026-09-28 with OpenAI’s open-source tiktoken library, encodings cl100k_base and o200k_base.

• Cost tests: run by The Wrong Default on 2026-09-28 and 2026-09-29 (Pacific) against the OpenAI Responses API (gpt-6-astra, web search enabled, default settings), each on a dedicated API key used only for that test. Per-run costs computed from OpenAI’s published rate card; invoices ($1.75 for the five-run test, $5.62 for the six-run test) matched our line-item computation to the cent.

• Bai et al., “How Do AI Agents Spend Your Money?” arXiv 2604.22750 (2026-04): https://arxiv.org/abs/2604.22750

• Kapoor et al., “Holistic Agent Leaderboard,” arXiv 2510.11977 (ICLR 2026): https://arxiv.org/abs/2510.11977

• He / Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference” (2025-09-10): https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/

• Intercom Fin pricing and outcomes: https://fin.ai/help/en/articles/13975800-fin-pricing-outcomes

• Zendesk, automated resolution tiers: https://support.zendesk.com/hc/en-us/articles/9570369117338-About-automated-resolution-tiers

• Deloitte DART, outcome-based pricing in agentic AI (2026-06-04): https://dart.deloitte.com/USDART/home/publications/deloitte/industry/technology/accounting-outcome-based-pricing-agentic-ai

• Gartner via BusinessWire (2024-10-21): https://www.businesswire.com/news/home/20241021957700/en/Gartner-Identifies-Four-Emerging-Challenges-to-Delivering-Value-from-AI-Safely-and-at-Scale

• IDC, J. Jensen (2025-12-01): https://www.idc.com/resource-center/blog/balancing-ai-innovation-and-cost-the-new-finops-mandate/

From the newsletter to the work

The Wrong Default names the decision nobody owned. Fellowship Intelligence helps you own it.

Thomas Tornatore founded Fellowship Intelligence, an AI strategy, governance, and leadership firm for organizations where a wrong AI-influenced call carries real weight.