Bias Testing
The process of testing an AI model for accuracy and outcome differences across groups, including protected classes and proxy variables.
Bias testing, also called fairness testing, is the process of checking whether an AI model produces systematically different outcomes across groups. It usually includes accuracy testing, disparate impact analysis, and protected-class proxy screening.
The NAIC Model Bulletin and state rules such as NYDFS Circular Letter No. 7 require documented bias testing for high-risk AI systems. The testing must usually be done before deployment and repeated periodically, especially after model updates, retraining, or changes in the data environment.
A complete bias-testing program records the groups tested, the metrics used, the results, and any actions taken. See our glossary entries on algorithmic bias, disparate impact, and the proxy test.