Skip to content
Implement
Archive

Enterprise AI Cost Basics

A practical primer on where enterprise AI costs accumulate and how leaders should think about them.

5 min read
AI economicscost managementstrategy
Archived. This piece has not been reviewed against sources since it was published and is not part of the maintained library. Figures and examples in it may be out of date, and some examples are illustrative rather than real cases. It is kept so that existing links still work. See the maintained library.

Key takeaways

  • The most visible AI cost is usually the model invoice, but the largest costs often sit in infrastructure, people, integration, and governance.
  • IDC and FinOps Foundation data suggest many organisations are still undercounting AI cost because spend is fragmented across budgets and operating layers.
  • A useful cost model distinguishes shared platform investment from local demand and updates as a service matures.
  • Leaders should use AI cost basics as an entry point into the seven-category AI TCO Framework, not as a substitute for it.

Why this matters

AI spending rarely shows up in one place. Model usage, infrastructure, orchestration, data work, governance, and oversight often land in different budgets. Interpretation IDC's research suggests a large share of AI tool spending still happens outside formal IT budgets. Interpretation FinOps Foundation shows AI spend management has become widespread operational work. Interpretation Together, those points explain why many executive teams still struggle to answer a simple question: what does this capability really cost to run well? Interpretation

By May 2026, corporate America had begun rationing AI access (paywalled) as costs outpaced budgets-a market signal that incomplete cost visibility was failing at scale.

Discovery at scale: In 2026, Marc Benioff said Salesforce expected to spend about $300 million on Anthropic tokens in 2026, largely for coding-related work. The spend was fragmented across departments, embedded in SaaS contracts, and categorised inconsistently across finance systems.

A particular challenge is that SaaS-based AI tools often bundle and obscure underlying token consumption within subscription pricing, making it difficult to understand true unit economics or identify optimisation opportunities. Interpretation For more on this visibility gap, see SaaS Token Opacity: The Hidden Economics of AI Subscriptions.

The seven-layer cost taxonomy

The canonical AI TCO Framework uses a seven-layer cost taxonomy that captures where enterprise AI costs actually accumulate: Interpretation
  1. Infrastructure - accelerated compute, GPU capacity, storage, networking, runtime services
  2. Data and context - data ingestion, retrieval pipelines, vector systems, indexing, metadata
  3. Models - API tokens, SaaS AI premiums, reserved capacity, fine-tuning
  4. Integration and workflow redesign - application design, system integration, workflow changes
  5. People and capability - platform engineers, data specialists, governance labour, support staff
  6. Governance, safety, and compliance - evaluation, red teaming, policy, auditability, security
  7. Operations and portfolio oversight - monitoring, incident response, allocation, portfolio review
This taxonomy is deliberately granular because each layer requires different measurement approaches, ownership models, and optimization strategies. Interpretation
Seven cost layers of enterprise AI, with model inference as the visible slice above the meter
The seven layers above. The model invoice is the part most organisations see first.
Seven cost layers of enterprise AI, with model inference as the visible slice above the meter

The seven layers above. The model invoice is the part most organisations see first.

What strong cost documentation should do

Good cost documentation should make it easier to: Interpretation
  • document where spend originates
  • connect technical activity to economic units
  • compare use cases with a consistent cost frame
  • support better investment, prioritisation, and governance decisions
The goal is not just to report spend, but to make AI capability economically legible. Interpretation

When cost opacity kills value: Starbucks retired its inventory management AI agent after discovering the system's recommendations were being routinely overridden by store managers who understood local context the model missed. The agent consumed resources but delivered no measurable improvement over human judgement. The retirement decision came only after establishing baseline performance metrics and full-stack cost accounting.

Common cost blind spots

Organisations consistently undercount AI costs in three areas: Interpretation

Embedded AI tiers in SaaS contracts. Many enterprise software vendors now include AI features as premium tiers or add-ons, but the AI-attributable cost component is rarely stated explicitly in contracts. Interpretation Finance teams often categorise these as standard software spend rather than AI investment, creating a systematic undercount. Interpretation

Duplicate tooling across departments. Without central visibility, organisations frequently purchase multiple tools serving the same function - general AI assistants, coding copilots, or document processing systems - across different business units. Interpretation Even tier-1 enterprises struggle with this-Uber's experience of exhausting annual AI budgets in four months illustrated how fragmented spend and consumption can outpace both planning and visibility systems.

The adoption gap: Market observations in early 2026 indicated significant enterprise Copilot subscription cancellations, with organisations citing low adoption and unclear value as primary reasons. The pattern: seats purchased on potential, cancelled on measured reality. Many organisations discovered they were paying for hundreds of seats with fewer than 40% active users.

Governance and safety costs. The NIST AI Risk Management Framework treats economic sustainability as a governance concern, not just a finance concern. Interpretation Responsible AI deployment requires investment in safety systems, alignment work, audit capabilities, and risk management - costs that often sit outside the AI project budget but are essential to sustainable operation. Interpretation

What to do next

For finance leaders:

  • Ask whether the current cost view includes labour, governance, and integration rather than only direct vendor fees.

  • Separate exploratory AI spending from operating AI services in reporting.

  • Establish a named owner for AI economics across the estate, with mandate from CFO or CIO. For technology and platform leaders:

  • Instrument the top AI workflows so cost can be tied to service behaviour, not only providers.

  • Expose shared platform cost explicitly before more local use cases are approved.

  • Set guardrails where consumption is unbounded - per-task and per-run budgets on any autonomous agent, with automatic halt. For operating-model leaders:

  • Use this primer as the opening layer, then move into the seven-category AI TCO Framework, FinOps for AI, and TBM for AI for the fuller model.

  • Build attribution coverage as the lead metric - the percentage of AI spend that can be connected to measured outcomes.

Where cost basics fit in the maturity model

The five-rung distinction ladder describes AI economics maturity: activity, adoption, productivity, value, strategic impact. Interpretation Understanding enterprise AI cost basics addresses the bottom rung - activity - by establishing what is being used and what it costs. Interpretation This is necessary but not sufficient. Knowing what AI costs is different from knowing whether it delivers value. Interpretation BCG's research on the AI value gap shows that organisations with early measurement discipline close the gap faster than those that wait for perfect data. Interpretation

The optimist's case

Cost visibility is the foundation for value management. You cannot optimise what you cannot measure. Early cost discipline prevents the budget overruns that force reactive rationing. The FinOps Foundation's guidance on AI value management emphasises that early action prevents later crisis.

The sceptic's case

Cost accounting without value measurement produces cost-cutting theatre, not strategic decision-making. Organisations that focus on cost basics often stop there, never building the productivity baselines or value attribution systems that matter. The risk is that cost visibility becomes the goal rather than the starting point.

Synthesis · house view

Cost basics are necessary but deliberately incomplete. The goal is minimum viable economic visibility to prevent budget crises while building the data foundation for productivity measurement, value realisation, and strategic impact assessment. Organisations should spend 30 days on cost basics, then move immediately to adoption and productivity measurement.

References and further reading

  1. FinOps Foundation, FinOps for AI: Scopes and Capabilities, 2025
  2. BCG, The Widening AI Value Gap: Build for the Future, 2025
  3. BCG, From Potential to Profit: Closing the AI Impact Gap, 2024
  4. NIST, AI Risk Management Framework, 2023
  5. IDC, Worldwide AI Spending Guide, 2025
  6. AWS, Closing the AI Value Gap, 2024
Explore next

Continue exploring

Follow the threads that connect AI cost, value, governance, and operating discipline.

AI TCO Framework

The seven-layer framework for understanding where enterprise AI costs actually accumulate.

Glossary

Shared definitions for AI cost, inference, TCO, ROI, FinOps, and governance language.

AI Economics Maturity Model

A maturity model for building from cost awareness toward governed, provable AI value.

SaaS Token Opacity

How subscription pricing obscures underlying token consumption and creates visibility gaps in AI economics.

By Ed Epstein, who works at IBM. The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. About and disclosure · Methodology