Measurement capture is one of the structural reasons that AI proof gaps persist even in organisations with good governance intentions. It is not primarily a problem of dishonesty — it is a problem of evaluation design.
When the team that built an AI system is asked to measure its value, they face a series of methodological choices that are genuinely ambiguous: which baseline period to use, which metrics to prioritise, how to attribute improvement to the AI versus to other changes happening concurrently, how to handle adoption variance across users. Each of these choices has a range of defensible options. Confirmation bias — the well-documented human tendency to find the evidence that supports what we already believe — pushes evaluators toward the choices that produce more favourable results.
The practical implication is that AI value claims produced by the teams that built the AI system should be treated as indicative rather than definitive, particularly for material investment decisions. Independent evaluation — whether through internal audit, a separate analytics function, or an external reviewer — provides a more reliable evidence base.
Organisations that invest in measurement independence earlier will find that their AI portfolio decisions improve, because the evidence base is more trustworthy. Those that do not will find that they are eventually making allocation decisions based on self-reported performance data from teams with a structural reason to report positively.
For the governance implications, see The AI Value Gap.