Watch: the module in 1:14
Read the transcript
Once AI capacity is shared, someone has to decide who pays for it, including the hours it sits idle.
Start with one pool each month: compute, serving software and the platform team. Pooled cost always equals allocated cost plus unallocated cost. Never spread the remainder silently.
Idle capacity has three causes, each with its own owner. Headroom held for a service's peak belongs to that service. A reservation nobody used belongs to the team that booked it. Capacity nobody reserved belongs to the platform owner.
Spread idle cost across users and the internal rate climbs. Once it passes the external price, teams leave and utilisation falls further.
Pull the buying levers in order of commitment: caching, then batch, then committed capacity. Commit for the floor of demand, not the peak.
In the exercise, you will allocate a month of GPU cost by hand, keep idle visible and owned, and test a commitment by its break-even utilisation.
By the end of this module you can
- Build a monthly cost pool for shared AI capacity and allocate it with a fair driver.
- Split idle capacity by cause and assign each cause an owner.
- Sequence the buying levers (caching, batch, commitment) and test a commitment by break-even utilisation.
- Map AI charges to FOCUS 1.4 columns and explain where FinOps hands over to value management.
FinOps and TBM lens: this is your core module. Finance lens: note the showback to chargeback conditions.
The argument in brief
Once capacity is reserved or owned, its cost is largely fixed while demand from the teams using it moves by hour and by project. The question becomes who pays, including for the hours it sits idle. Technology business management and IT financial management have solved versions of this for years. AI adds an expensive unit, uneven demand, and an external model that is always one configuration change away.
This module covers the allocation rules, the buying levers that set most of what a team pays per token, and how AI charges can be reported in FOCUS before the specification has dedicated token columns. It ends with the boundary that matters for the rest of the course: FinOps aims at business value, but billing data does not contain the baseline, quality floor or evidence of change that a value claim needs.
Key ideas
- Pooled cost equals allocated cost plus unallocated cost, every month. Never spread the unallocated part silently.
- Idle capacity has three causes with three owners: headroom held by policy, reservations not used, and capacity nobody reserved.
- Spreading idle cost raises the internal rate. If it passes the external price, teams leave and utilisation falls further.
- Pull levers in order of commitment: caching, then batch, then committed capacity. Commit for the floor, not the peak.
- Start with showback. Move to chargeback when the driver is reliable, the pool reconciles, teams can control use and the rate has been compared with external prices.
Terms to know
- Cost Pools
- Aggregated groups of cost that are collected before being allocated, mapped, or analyzed further.
- Cost Allocation
- The method used to distribute shared technology cost across services, teams, products, or business units.
- Idle Capacity
- Reserved or owned AI compute that is paid for but not doing productive work in a period, whether held as headroom, booked but unused, or never allocated.
- Showback
- A mechanism for reporting technology cost back to consumers without formally charging them for it.
- Chargeback
- A mechanism for allocating and billing technology cost directly to the business unit, service owner, or consumer responsible for demand.
- Shared AI Platform
- A common internal platform that provides reusable AI services, tooling, guardrails, and infrastructure to multiple teams.
- FOCUS
- The FinOps Open Cost and Usage Specification, an open standard for normalising technology billing and usage data.
- FinOps Scope
- A formal domain of technology spend governed within the FinOps framework, such as cloud, SaaS, or AI.
Read
In this order. Each opens on the main site.
- 1Allocating Shared AI CapacityPools, drivers, idle capacity and internal pricing tiers.18 min
- 2Buying AI Capacity and Reporting It in FOCUSCaching, batch, commitments and a FOCUS 1.4 mapping.9 min
- 3FinOps and AIThe practice that makes AI spend visible, allocated and efficient.12 min
- 4FinOps for AI Is Necessary, Not SufficientWhere cost management hands over to value management.20 min
Worked example: commit for the floor, not the peak
Invented figures. Capacity that would cost £100 an hour at pay-as-you-go prices if it ran flat out is offered at £60 an hour on commitment. A customer-facing workload is busy for ten hours a day and quiet overnight.
| Break-even | £60 ÷ £100: commitment pays only above 60% average utilisation. |
|---|---|
| Actual use | The workload averages 35%, ranging from 10% overnight to 90% at midday. |
| Commit for the peak | Pays £60 an hour for use worth £35 at pay-as-you-go prices. |
| Commit for the floor | Commit the 10% the workload never drops below: £6 an hour for use worth £10. Pay per request for the rest. |
The result
Committing for the peak loses money on every quiet hour. Committing for the base load, and buying the rest on demand, saves money at any level of use above the floor.
Test every commitment against its break-even utilisation, and apply caching and batch before you commit.
From Buying AI Capacity and Reporting It in FOCUS, where the figures are explained in full.
Try it: allocate a month of shared GPU cost
A shared platform runs four GPUs for a month: 2,920 GPU-hours. The full pool, including the platform team and serving software, is £60,000. Team A reserves one GPU (730 GPU-hours) for a customer-facing assistant. Team B uses 876 GPU-hours of metered capacity. Overnight batch jobs use 584 GPU-hours on a tier priced at half the rate. 365 GPU-hours are held as headroom for Team A's peak. The rest was neither reserved nor used.
| Pool and GPU-hours | £60,000 and 2,920 |
|---|---|
| Team A reservation | 730 GPU-hours |
| Team B metered use | 876 GPU-hours |
| Batch use at half rate | 584 GPU-hours |
| Headroom for Team A | 365 GPU-hours |
- Work out the fully loaded rate per GPU-hour.
- Charge Team A, Team B and the batch jobs.
- Show the remaining lines (headroom, unallocated idle and the batch discount), each with an owner, and check that the pool reconciles.
- Work out the rate Teams A and B would face if those remaining lines were spread across them instead.
- Finally: capacity that would cost £100 an hour at pay-as-you-go prices is offered at £60 an hour on commitment. What average utilisation does commitment need to break even?
Show the worked answer
Rate: £60,000 ÷ 2,920 is about £20.55 a GPU-hour.
Team A £15,000 (charged whether used or not). Team B £18,000. Batch £6,000, with £6,000 shown as the batch discount.
Headroom: 365 hours, £7,500, owned by Team A's service owner (or a central budget if resilience is shared by decision). Unallocated idle: 2,920 − 730 − 876 − 584 − 365 = 365 hours, £7,500, owned by the platform owner. £39,000 charged + £7,500 + £7,500 + £6,000 = £60,000.
Spread across A and B: (£60,000 − £6,000 batch) ÷ 1,606 hours is about £33.62 a GPU-hour, roughly 64% higher for the same use.
Commitment breaks even at £60 ÷ £100 = 60% average utilisation. Commit for the base load, not the peak.
Keep idle cost visible and owned. A persistent unallocated line is a sourcing signal, not a rounding error.
Check your understanding
Five practice questions. The final exam uses different questions in the same style.
Practice only. Nothing is recorded.
Progress is saved in this browser only.