Model risk was formalised as a regulatory concept in financial services through guidance that requires banks and other regulated firms to maintain robust model validation and oversight processes for any model used in material decisions. It has since expanded in relevance to healthcare, insurance, and increasingly to any sector where AI systems make or inform significant decisions.
Model risk comprises several distinct sub-risks. First, model errors — the model produces incorrect outputs due to flawed design, inadequate training data, or distributional shift between training and deployment conditions. Second, misuse — a model is applied to a context it was not designed for, or used in a way that exceeds its validated scope. Third, governance failure — the model operates without adequate monitoring, validation, or oversight, and errors go undetected.
For enterprise AI buyers, model risk has direct economic implications. In regulated industries, model risk management is a compliance obligation with associated costs: independent model validation, ongoing monitoring, documentation, and remediation when issues arise. These costs are persistent, not one-time, and tend to grow as the model is more widely used in more consequential decisions. Standard AI TCO frameworks systematically underestimate model risk management costs, particularly in regulated contexts.
In less-regulated industries, model risk remains a material concern even without formal regulatory requirements. An AI system that makes systematically incorrect decisions in a customer-facing context — credit scoring, product recommendations, medical triage support — can create liability, reputational damage, and remediation costs that dwarf the system's annual operating budget.
For the full analysis of model risk in regulated industry AI economics, see AI Economics in Regulated Industries.