Start with the distinction ladder
Before you choose metrics, place each one on the distinction ladder. Every AI metric belongs on one of five rungs: activity, adoption, productivity, value, or strategic impact. The higher the rung, the harder the metric is to game, and the more it matters. Activity and adoption metrics are necessary for operations, but they do not prove value. The rest of this page focuses on rungs 3 to 5.
The five rungs
Every AI metric sits on one of five rungs. The rungs are ordered by how close they are to business value. The higher the rung, the harder the metric is to game, and the more it matters.
Rung 1: Activity
Things happening. Tokens consumed, API calls made, models deployed, users registered. Activity metrics tell you the system is being used. They do not tell you if it is useful.
Example: 2.4 million tokens consumed this month
Rung 2: Adoption
People using it. Active users, session frequency, feature usage. Adoption metrics tell you people are trying it. They do not tell you if it is working.
Example: 340 active users, 68% weekly return rate
Rung 3: Productivity
Work done faster. Time saved, tasks completed, throughput increased. Productivity metrics tell you the work is getting done quicker. They do not tell you if the work matters.
Example: 4.2 hours saved per user per week
Rung 4: Value
Business outcomes improved. Revenue increased, costs reduced, quality improved, risk lowered. Value metrics tell you the business result changed. They do not tell you if it was strategic.
Example: £840k in labour cost avoided, 12% reduction in processing errors
Rung 5: Strategic impact
Capability built, position strengthened, options created. Strategic metrics tell you the organisation is more capable than it was. They are the hardest to measure and the most important.
Example: New product capability enabled, competitive position defended, regulatory risk reduced
The ladder is not a judgement. Activity metrics are not bad. They are just on the first rung. The problem is when organisations treat first-rung metrics as if they were fourth-rung metrics, or when dashboards mix all five rungs together without distinction.
The risk track
There is a parallel track for risk metrics. They run alongside the value ladder, but in the opposite direction: the higher the risk metric, the more it threatens value.
- Risk exposure: What could go wrong (model bias, data leakage, compliance breach)
- Risk events: What did go wrong (incidents, failures, audit findings)
- Risk impact: What it cost when it went wrong (financial loss, reputational damage, regulatory penalty)
Risk metrics belong on the ladder too. A dashboard that only shows value metrics without risk metrics is incomplete. The full picture is value minus risk.
Where common metrics sit
Here is where the most common AI metrics sit on the ladder:
| Metric | Rung |
|---|---|
| Tokens consumed | Activity (1) |
| API calls made | Activity (1) |
| Models deployed | Activity (1) |
| Active users | Adoption (2) |
| Session frequency | Adoption (2) |
| Time saved per task | Productivity (3) |
| Tasks completed faster | Productivity (3) |
| Cost per successful outcome | Value (4) |
| Revenue attributed to AI | Value (4) |
| Quality improvement | Value (4) |
| New capability enabled | Strategic (5) |
| Competitive position defended | Strategic (5) |
The diagnostic question
The distinction ladder gives you one diagnostic question to ask of any AI dashboard:
Which rung is each metric on, and why does it matter?
If the dashboard cannot answer that question, it is a vanity dashboard. It is showing activity and calling it value. The ladder forces the distinction.
The lead metric: attribution coverage
If you can only track one metric across your AI portfolio, track attribution coverage. It is the percentage of your total AI cost that can be linked to a measured business outcome.
Attribution coverage = (Cost attributed to measured outcomes) / (Total AI cost)
The unattributed remainder is the value gap. One number to lead with, one gap to close.
Attribution coverage is a lead metric because it predicts value realisation. If you cannot attribute cost to outcome, you cannot prove value. If you cannot prove value, you cannot defend the spend. Coverage is the early warning.
Maturity: Emerging
Requires: cost visibility by initiative, outcome measurement for major use cases, attribution logic to link cost to outcome.
Capture status: from potential to sustained
Not all claimed value is equal. Every value item should be tagged with its capture status to distinguish modelled opportunities from realised benefits.
Interpretation Most AI value claims stop at “potential” or “observed”. Few reach “captured”. Almost none track “sustained”. This progression explains why portfolio-level ROI remains elusive despite impressive pilot results.
The comprehensive metric set
The full reference set contains 42 metrics across 12 categories. Each metric includes: definition, what it is good for, where it fails, and what maturity level you need to use it. Below are the key metrics from each category.
Portfolio Coverage
Attribution coverage
Definition: Percentage of total AI cost linked to measured business outcomes
Good for: Portfolio-level value proof, board reporting
Failure mode: Can be gamed by attributing cost to weak outcomes
Initiative coverage
Definition: Percentage of AI initiatives with defined success metrics
Good for: Governance maturity, ensuring clear goals
Failure mode: Metrics can be defined but not measured
Maturity: Foundational
Cost Efficiency
Cost per successful outcome
Definition: Total AI cost divided by number of successful outcomes delivered
Good for: Unit economics, comparing efficiency across use cases
Failure mode: Requires clear definition of ‘successful outcome’
Fully loaded cost per use case
Definition: Total cost including infrastructure, operations, governance, not just inference
Good for: True TCO visibility, portfolio prioritisation
Failure mode: Requires cost allocation methodology
Maturity: Systematic
Productivity Impact
Realised productivity gain
Definition: Measured output increase or cost reduction, not self-reported time savings
Good for: Honest productivity measurement, avoiding self-report bias
Failure mode: Requires baseline data and control groups
Maturity: Systematic
Value Realisation
Multi-dimensional value
Four dimensions: Revenue growth, cost reduction, quality improvement, risk mitigation
Good for: Capturing full value, avoiding single-dimension optimisation
Failure mode: Requires weighting across dimensions
Maturity: Systematic
ROI and Payback
Simple ROI
Definition: (Value delivered - Cost) / Cost, expressed as percentage
Good for: Quick business case assessment
Failure mode: Ignores time value of money and strategic value
Maturity: Foundational
Risk and Compliance
Model risk coverage
Definition: Percentage of production models with completed risk assessments
Good for: Governance maturity, regulatory readiness
Failure mode: Assessment completion does not equal risk mitigation
Note: The full reference set includes 42 metrics across 12 categories. See the complete AI Economics KPIs library for the full set with filters by role, maturity level, and governance domain.
Economic yield families
Economic yield measures compare worthwhile outcomes with fully loaded cost, risk and operational burden. There is no single universal yield metric. Use the family that matches your value dimension.
Workflow yield
Measured operating benefit / full workflow cost
Use for: Claims processing, support resolution, document review, software delivery, forecasting
Revenue yield
Incremental contribution margin / full AI cost
Use when AI changes: Conversion, retention, pricing, product adoption, sales capacity
Risk yield
Expected loss avoided / full AI cost
Use for: Fraud, cyber, compliance, safety, operational resilience
Capacity yield
Productive capacity redeployed / full AI cost
Note: Stricter than hours saved. Time has value only when it changes output, cost, service or strategic capacity.
Quality-adjusted yield
Successful outcomes × quality weight / full cost
Use when: Output quality varies significantly and affects downstream value
Warning: Do not compare unlike outcomes using one league table. A medical decision, code completion and fraud alert should not be ranked by a single yield score.
Behavioural metrics
Behavioural outcomes determine whether AI outputs become useful work. These metrics track trust, adoption quality, review burden and capability effects.
Appropriate-use rate
Share of AI use occurring in tasks where evidence shows net benefit
Trust calibration gap
Difference between user confidence and actual performance
Review debt
Backlog or superficial approval when outputs are generated faster than validation capacity
Rework displacement
Whether AI removes rework or moves it downstream
Learning transfer
Whether users improve independent task performance over time
Shadow-use exposure
Estimated use outside approved systems or evidence standards
For a complete behavioural measurement framework, see The Behavioural P&L of AI.
Agentic demand metrics
Agentic systems create machine-generated demand that is not limited by headcount or working hours. These metrics track autonomous consumption and value per objective.
Autonomous calls per human request
Model calls generated by agents per initial user objective
Cost per completed objective
Full cost including retries, tool calls and verification per successful agent objective
Retry rate
Frequency of agent retries and correction loops
Idle loop rate
Agent calls that produce no useful outcome or forward progress
Value at risk from runaway demand
Potential cost exposure if agent demand expands without governance
Interpretation Agentic demand can scale faster than value. Track cost per completed objective, not cost per call, to avoid rewarding activity over outcomes.
The instrumentation gap
In our reading of the published surveys and case material, measurement gets thinner as you climb. Activity and adoption (rungs 1-2) are widely measured. Productivity (rung 3) is measured less often, value (rung 4) less often still, and strategic impact (rung 5) rarely. The direction is clearer than the exact proportions.
The gap is not conceptual. It is instrumentation. The systems that track AI usage do not connect to the systems that track business outcomes. The token meter does not talk to the P&L. The user analytics do not talk to the CRM. The model logs do not talk to the quality system.
Closing the instrumentation gap requires integration work, not just metric definition. You need to connect AI telemetry to business telemetry. That is an engineering problem, not a measurement problem.
Short self-audit
Use this checklist to audit your metric selection. Answer honestly.
Metric selection checklist
Goodhart’s Law warning
Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. Every metric in this framework can be gamed. The question is not whether gaming is possible, but whether the cost of gaming exceeds the cost of honest measurement.
The tokenmaxxing leaderboards collapsed in early 2026 because they optimised for the wrong thing. Organisations competed to minimise token cost per task, which incentivised shorter outputs, simpler prompts, and cheaper models. The result was lower quality, not lower cost. The leaderboards were gaming the metric, not improving the outcome.
The defence against Goodhart’s Law is to measure multiple dimensions, track failure modes, and retire metrics when they stop being useful. No single metric is sufficient. The portfolio of metrics matters more than any individual metric.
Related reading