Once a team has chosen a model, three buying decisions set most of what it pays per token: whether repeated context is cached, whether work that can wait is sent as a batch, and whether steady demand is bought as committed capacity rather than per request. They are worth taking in that order, because each one asks for more commitment than the last.
The second half of this piece covers reporting. AI charges are arriving in FinOps data now, before the FOCUS specification has dedicated token columns. A consistent mapping to the current columns avoids rebuilding reports later.
Lever one: prompt caching
Most production prompts repeat a large block of context: system instructions, a policy document, a long conversation history. Caching stores that block so later requests read it at a discount instead of paying for it again.
Caching needs no commitment and usually no change to how the application behaves, so it comes first. Its value depends on how much of each prompt is repeated and how often the cache is hit before it expires. A cache that is written and rarely read costs more than no cache at all.
Lever two: batch processing
Work that does not need an answer in seconds, such as overnight classification, document backlogs, evaluations and report generation, can be sent as a batch.
The work here is in the application, not the contract: separating requests that can wait from those that cannot. For many back-office workloads that split is large, and the discount applies to every token in it.
Lever three: committed capacity
Each major cloud sells model capacity by the hour rather than by the token:
| Offer | Unit | Terms | Billing |
|---|---|---|---|
| Microsoft Foundry provisioned throughput | Provisioned throughput units (PTUs) | Hourly, or an Azure Reservation for one month or one year | Billed for deployed PTUs whether or not they are used |
| Amazon Bedrock Provisioned Throughput | Model units | No commitment, one month or six months; longer terms cost less per hour | Hourly, until the capacity is deleted |
| Google Vertex AI Provisioned Throughput | Generative AI scale units (GSUs) | One week, one month, three months or one year | Fixed-term subscription |
Microsoft's documentation makes two points that apply to all three. A reservation is a billing discount, not a guarantee of capacity, so deployments should be created first and reserved afterwards. And capacity bought for peak demand is paid for at the trough.
Commitments also tie the buyer to a supplier's balance sheet for the term. That matters more for specialist GPU providers than for the major clouds: Gartner expects more than 80% of neocloud providers to stop operating independently by 2030 (Gartner, 14 October 2026). See What a Supplier Write-Down Cycle Means for AI Buyers. At the major clouds the near-term issue is scarcity rather than failure: with AWS capacity reportedly largely sold out into 2027, committed discounts may be harder to win, and because a reservation is a discount rather than a guarantee, buyers should confirm the capacity exists before they commit. The article's update of 5 November 2026 sets out the evidence and its limits.
When commitment pays
The test is utilisation. Compare the committed price per hour with what the same capacity would cost per hour at pay-as-you-go prices if it were fully used:
Break-even utilisation = committed cost per hour
÷ pay-as-you-go cost of the capacity at full use, per hour
Reporting AI charges in FOCUS
FOCUS 1.4, ratified on 4 June 2026, has no column for token type or model identity. The FinOps Foundation says FOCUS 1.5 is scoped to add model identity and input and output token consumption. Until then, AI charges can be carried in existing columns. The mapping below is a working convention, not part of the specification.
| What the AI bill shows | FOCUS 1.4 column | Notes |
|---|---|---|
| Model and token type (input, output, cached, batch) | SkuId, SkuMeter, ChargeDescription | Keep one row per model and token type. Batch and cached tokens usually appear as their own price lines rather than as a discount |
| Number of tokens | ConsumedQuantity, ConsumedUnit | Use one unit name, such as Tokens, across providers |
| Billing unit | PricingQuantity, PricingUnit | Often per million tokens; keep it distinct from the consumed unit |
| Who provides the model service | ServiceProviderName | The spec's marketplace and reseller rules decide this for models bought through a cloud platform. Record the decision and apply it consistently |
| Whose infrastructure runs it | HostProviderName | Must match the service provider unless the host is chosen by the customer or exposed by the provider |
| Who sends the invoice | InvoiceIssuerName | Can differ from both of the above |
| Category | ServiceCategory, ServiceSubcategory | AI and Machine Learning, with Generative AI as the subcategory for model usage |
| Project, workspace or API key | SubAccountId, ResourceId, Tags | The allocation key. Without it, AI spend cannot be shown back to the teams that use it |
| Reservation or capacity purchase | ChargeCategory (Purchase), CommitmentDiscountId or CapacityReservationId | Show the purchase once, and the usage it covers against it |
| What the organisation actually pays | BilledCost, EffectiveCost | Effective cost spreads commitment purchases over the usage they cover, so unit costs stay comparable |
Where a provider does not publish a FOCUS export, the usage and cost APIs listed in the billing APIs reference can be converted with the same mapping. When FOCUS 1.5 is ratified, the model and token-type fields above move into their dedicated columns, and the rest of the mapping should stand.
What would change this view
- If providers move to outcome-based or blended pricing, token-level levers matter less and the mapping above needs a different unit.
- If committed capacity becomes portable across model generations, as Microsoft's model-independent reservations already are within a deployment type, the risk of committing falls and the break-even utilisation matters less.
Sources
- Anthropic, prompt caching and batch processing documentation, checked 16 October 2026.
- OpenAI, Batch API guide, checked 16 October 2026.
- Google, Gemini API batch mode and Vertex AI consumption options, checked 16 October 2026.
- Microsoft, provisioned throughput billing and cost management, checked 16 October 2026.
- AWS, Provisioned Throughput in Amazon Bedrock, checked 16 October 2026.
- FinOps Foundation, Introducing FOCUS 1.4 and the FOCUS 1.4 specification.