Key takeaways
- Model inference fees are the most visible AI cost but typically represent only ten to twenty per cent of total enterprise AI programme cost, where the denominator includes integration, governance, support, and platform costs (see the note below on denominators).
- Inference costs vary by several orders of magnitude depending on model choice, deployment pattern, and workload design. A well-optimised AI workflow can cost ten to fifty times less than a naively assembled one using the same underlying model.
- In our assessment, integration and maintenance engineering is the largest hidden cost for most organisations in the first two years of AI deployment.
- Governance overhead is real, measurable, and frequently underbudgeted. It typically runs at fifteen to thirty per cent of the operational AI investment in regulated environments.
What "ten to twenty per cent" is a share of
The ten to twenty per cent figure is the site's own estimate, and it is a share of total enterprise AI programme cost: model inference plus integration engineering, orchestration, governance, support, and platform costs. It is an interpretation, not a measured benchmark.
It should not be compared directly with commentary that reports inference as the majority of AI compute or infrastructure spend. That commentary uses a narrower denominator (the infrastructure bill, where inference naturally dominates training and idle capacity) and often a different population, such as AI-native companies rather than enterprise buyers. Both statements can be true. Check the denominator before using either.
Price ranges. The per-token ranges below are planning references, not procurement benchmarks. They were last verified on 15 October 2026 against the published price lists of Anthropic and Google. The earlier ranges ($3 to $30 frontier, $0.40 to $3 mid-tier, $0.05 to $0.40 small) pre-dated the current price lists and have been replaced.
Why AI cost benchmarks are difficult
AI costs are unusually hard to benchmark because the inputs vary enormously: different models, different deployment architectures, different workloads, different governance requirements, and different organisational maturity all produce cost structures that resist direct comparison.
Provider pricing also changes frequently, sometimes dramatically. What was a useful reference number in one quarter may be significantly different six months later as model efficiency improves, competition increases, or providers rebalance their pricing tiers. InterpretationInference costs
Inference is the cost of running a model to produce an output. It is the most visible line item in enterprise AI spend and the one most frequently used as a proxy for total AI cost. That proxy is misleading in most enterprise contexts, but understanding inference costs is still the right starting point. Interpretation
Per-token pricing (API-based) is the dominant commercial model for frontier and large language model usage. Current ranges (15 October 2026) vary substantially: Interpretation- High-capability frontier models: about $20 to $50 per million output tokens (Anthropic Opus 5.5 at $20 and Fable 5.1 at $50)
- Mid-tier models: about $3.75 to $10 per million output tokens (Google Gemini 3.8 Flash at $3.75, rising to $7.50 from 1 January 2027, and Anthropic Sonnet 5.5 at $10)
- Efficient small models: about $0.50 to $5 per million output tokens (Anthropic Haiku 5.5 at $0.50 for prompts up to 100,000 tokens, Google Gemini 3.5 Flash-Lite at $2.50 and Anthropic Haiku 4.5 at $5)
- Self-hosted open models: effective cost depends on utilisation. Our earlier estimate was $0.10 to $1.50 per million tokens at reasonable scale once infrastructure is included. This is an estimate and was not verified against a source
These are list prices from two providers, read from their published price lists on 15 October 2026, and they omit others such as OpenAI whose price page could not be read. Batch discounts, caching and enterprise agreements move real costs, so treat the ranges as planning references.
The small-model floor moved on 7 October 2026, when Anthropic launched Haiku 5.5 at $0.10 input and $0.50 output per million tokens, a tenth of Haiku 4.5's price. The gap between the cheapest and the most expensive model on one provider's list is now a hundredfold, which makes model choice a larger cost lever than it was. The same model also shows how context tiers work: once a prompt exceeds 100,000 tokens, the whole request is billed at $0.50 input and $2.50 output, five times the standard rate (Anthropic pricing). A workload that drifts above that threshold can multiply its bill without any change in volume.
Real-world cost variance: Uber spent its entire 2026 AI coding budget in four months (Fortune, May 2026). The Information reported per-engineer API costs of $500 to $2,000 a month; that range comes from a single source and is indicative. See the Uber case study for what the public record shows and what it does not.
Platform and orchestration costs
Most enterprise AI use cases are not direct API calls. They involve an orchestration layer - retrieval-augmented generation pipelines, agent frameworks, workflow automation, or AI application platforms - that carries its own costs.
Enterprise AI platform licensing (for products like Microsoft Copilot, Salesforce Einstein, ServiceNow AI, Workday AI) typically runs at $20–$60 per user per month for AI-augmented seat licences, often layered on top of existing platform fees. At scale, this becomes the dominant AI cost line for many organisations - a workforce of five hundred knowledge workers using AI-augmented productivity tools costs $120,000–$360,000 annually in licence fees before any customisation or integration work. Interpretation Orchestration infrastructure (LLM gateways, vector databases, embedding services, retrieval pipelines) typically adds thirty to sixty per cent on top of inference costs for production-grade RAG or agentic deployments. This layer is often underestimated because it does not appear on the model provider's invoice. InterpretationIntegration and engineering costs
Integration engineering is consistently the largest underestimated cost category in enterprise AI deployments. Interpretation Building an AI capability into a business workflow requires connecting it to data sources, access controls, existing applications, and output consumers - none of which is trivial.
Initial integration build: $50,000–$300,000 for a production-grade, single-use-case integration in a typical enterprise environment. The wide range reflects the complexity of the target workflow and the quality of the existing data infrastructure. Simpler document processing use cases sit at the lower end; complex multi-system integrations involving real-time data, compliance controls, and legacy system connections sit at the upper end. Interpretation Ongoing maintenance: fifteen to twenty-five per cent of the initial build cost per year, driven by model updates, prompt degradation, upstream system changes, and expanding scope. Organisations that treat AI integrations as one-time builds rather than maintained software assets consistently underperform on total cost of ownership. Interpretation Enablement and change management: often excluded from technical cost models but typically equivalent to thirty to fifty per cent of the engineering cost for any workflow-level AI deployment. Tools that require significant user behaviour change have higher realisation costs, not lower. InterpretationWhen deployment cost exceeds value: In May 2026 Starbucks retired Automated Counting, an AI inventory tool rolled out across its North American stores about nine months earlier, and returned to manual counts. Reports said the tool often miscounted or mislabelled items and required stores to rearrange back-of-house storage (The Next Web, citing Reuters, 22 May 2026). The costs that decided it were in store labour and workflow, not in the tool's licence.
Governance and oversight costs
Governance costs are real, measurable, and consistently underestimated - particularly in regulated environments. Interpretation Prompt and output review: depends heavily on the use case risk level. Low-risk applications (summarisation, drafting assistance with human review) add ten to fifteen per cent to operational cost. Medium-risk applications (analysis used in decisions, customer-facing outputs) add twenty to forty per cent. High-risk or regulated applications (financial recommendations, clinical applications, legal documents) can add fifty to one hundred per cent of the raw AI cost in oversight and review. Interpretation Audit and compliance documentation: $20,000–$100,000 per year for a moderate AI portfolio in a regulated industry, driven by the requirement to document model versions, prompt changes, output sampling, and incident records. This is not optional in financial services, healthcare, or government contexts. Interpretation Model risk management: organisations subject to SR 11-7 or equivalent model risk frameworks treat AI models as managed models, which adds a structured validation and ongoing monitoring cost. Initial validation: $30,000–$150,000 per model in scope. Annual monitoring: ten to twenty per cent of the initial validation cost. InterpretationSupport and operations
Production AI workloads require operational support that differs from conventional software because the failure modes are different. An AI workflow can degrade gradually - producing outputs of lower quality rather than failing visibly - which means monitoring requires evaluation of output quality, not just system availability. Interpretation
AI operations (AIOps/MLOps) overhead: five to fifteen per cent of the total AI investment per year for organisations with mature practices. Organisations without dedicated AI operations capability typically see this cost absorbed into engineering time at higher effective rates. Interpretation Incident response: AI incidents (hallucinations at scale, prompt injection, data leakage, unexpected output patterns) require different response capabilities than conventional software incidents. The cost of not having these capabilities is not in the operations budget - it appears in remediation, legal, and reputational recovery. InterpretationTotal cost of ownership: indicative ranges
The following ranges represent the total annual cost of ownership for enterprise AI at different scale and complexity levels, combining all layers above. Interpretation Proof-of-concept or limited pilot (one or two use cases, controlled environment, limited users): $80,000–$300,000 in year one, primarily engineering and platform costs. Inference costs are typically negligible at this scale. Interpretation Departmental deployment (one or two use cases in production, active user population, governance in place): $300,000–$1.2m annually in stabilised operation, depending on use case risk, regulatory environment, and user scale. Interpretation Enterprise platform deployment (multiple use cases, shared AI platform, governance function, portfolio oversight): $2m–$15m annually. At this level, platform fees and governance infrastructure dominate. Inference costs represent a smaller fraction of total spend than at earlier stages. InterpretationToken spend at scale: In 2026, Marc Benioff said on the All-In podcast that Salesforce expected to spend about $300 million on Anthropic tokens that year, largely for coding (reported by Technology Magazine). It is a statement of planned spend, not a measured figure, and it shows how large a single token line can become.
The optimist's case
These ranges give a planning team somewhere to start. Without an external reference, early AI budgets tend to cover the model invoice and little else, and the integration, governance and support lines appear only after approval. Even a wide range forces those lines into the first estimate.
The sceptic's case
Benchmark ranges this wide are nearly useless for actual planning. A ten-to-one variance in inference costs, a six-to-one variance in integration costs, and a seven-to-one variance in enterprise platform costs mean these numbers tell you almost nothing about what your organisation will actually spend. The structural differences between organisations are so large that cross-company benchmarks mislead more than they inform.
Synthesis · house view
The ranges are wide because the variance drivers are real and material. The value is not in using these numbers as budget targets but in understanding which factors drive variance - workload design, governance requirements, data infrastructure quality, platform choice - and using that understanding to build organisation-specific models. The alternative to imperfect benchmarks is not perfect benchmarks; it is no external reference at all, which consistently produces worse planning outcomes.
Implications for planning
Three observations are consistently useful for organisations building AI cost plans.
First, optimise workload design before optimising vendor selection. The difference between a well-designed and poorly-designed AI workflow is larger than the difference between any two commercial model providers at comparable capability levels. Invest engineering time in workload design early. Interpretation Second, plan for governance costs from the start, especially in regulated environments. The most common cost surprise in enterprise AI programmes is the governance overhead that appears in year two - after the initial deployment, when the organisation realises it needs to manage the growing portfolio seriously. Interpretation Third, model total cost of ownership over three years, not year one. Year-one costs are inflated by integration engineering. Year-two and year-three costs reveal the stabilised operating cost, which is the number that should drive portfolio and scaling decisions. InterpretationReferences and further reading
- Anthropic, pricing documentation, read 15 October 2026, https://platform.claude.com/docs/en/about-claude/pricing
- Google AI for Developers, Gemini API pricing, read 15 October 2026, https://ai.google.dev/gemini-api/docs/pricing
- Inferbase, "The Real Cost of Inference at Enterprise Scale: A 2026 Pricing Audit", 13 May 2026, https://inferbase.ai/blog/enterprise-llm-inference-pricing-2026
- Fortune, "Uber COO: AI spending on tokens like Claude Code is hard to justify", 26 May 2026, https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-claude-code/
- The Next Web, "Starbucks pulls its AI inventory tool nine months in", 22 May 2026, citing Reuters, https://thenextweb.com/news/starbucks-ai-inventory-retired-north-america
- Technology Magazine, "Salesforce plans to spend US$300m on Anthropic AI tokens", 2026, https://technologymagazine.com/news/salesforce-plans-to-spend-us-300m-on-anthropic-ai-tokens