AI Vendor Risk Assessment for Insurers (NAIC Checklist)

The National Association of Insurance Commissioners (NAIC) found that 55% of responding health insurers use third-party components in their AI systems, and 15% rely entirely on third-party AI solutions.¹ The number is not surprising. Insurers buy fraud-detection models, underwriting platforms, claims triage tools, and generative AI copilots the same way they buy other software. What is surprising is how many carriers still believe that outsourcing the AI also outsources the compliance obligation.

It does not. The NAIC Model Bulletin is explicit: insurers remain responsible for compliance with insurance laws even when they use AI developed or operated by a third party.² When a regulator walks in with an examination notice, the carrier is the one that must produce the documentation, explain the model, and demonstrate that the system did not produce unfairly discriminatory or inaccurate outcomes. The vendor’s office is not where the examiner will sit.

This article is a practical checklist for vendor risk assessment in insurance. It translates the NAIC Model Bulletin and related state guidance into questions you can ask before signing a contract, clauses you should insist on, and monitoring you should keep running after the deal is done.

What the NAIC Model Bulletin Actually Requires

The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted in December 2023, has now been adopted by roughly half the states. It does not create a new federal insurance law. It restates that existing insurance laws, including unfair trade practice and unfair discrimination prohibitions, apply fully to AI-supported decisions.²

For third-party AI, the bulletin requires several specific things:

1. Due diligence. The insurer must assess the third party, its data, and its AI systems before deployment to ensure the system will not produce adverse consumer outcomes.
2. Contract controls. Contracts must preserve the insurer’s right to audit the vendor and require the vendor to cooperate with regulatory investigations related to the AI system.
3. Ongoing monitoring. The insurer must monitor the system after deployment, including for drift, bias, and accuracy degradation.
4. Documentation. The insurer must maintain records of all of the above and be ready to produce them to regulators.³

The New York State Department of Financial Services (NY DFS) applied a similar principle in a specific context with Insurance Circular Letter No. 7 in July 2024, requiring insurers to demonstrate that AI systems and external consumer data used in underwriting and pricing do not produce unfairly discriminatory or inaccurate outcomes.⁴ Colorado’s amended Regulation 10-1-1, which took effect on October 15, 2025, extends the same governance and risk management framework to health insurers and auto insurers.⁵

The message is consistent across jurisdictions: you can buy the tool, but you cannot buy your way out of the responsibility for what it does.

The Pre-Signing Vendor Assessment Checklist

A good vendor assessment is risk-based. A vendor whose AI merely drafts marketing copy needs less scrutiny than one whose model recommends coverage denials or sets prices. Sort your vendors by risk first, then apply the following questions. For high-risk vendors, every question below should be answered in writing before contract signature.

How to classify risk. A simple three-tier model works for most insurers:

• High risk: The AI can directly influence an adverse consumer outcome, such as a coverage denial, a premium increase, a claims payment decision, or a life insurance underwriting declination. These vendors require the full checklist, legal review, and board-level reporting.
• Medium risk: The AI supports operational decisions that affect consumers indirectly, such as fraud detection triage or customer service routing. These vendors require most of the checklist and periodic re-assessment.
• Low risk: The AI is used for internal productivity, such as drafting emails or summarizing documents, with no direct consumer impact. These vendors require basic security and privacy due diligence but not model-level validation.

This classification should be documented and revisited at least annually, or whenever the vendor’s system is expanded to a new use case.

Model and Data

1. What is the model’s intended use case, and has it been validated for that specific use case in insurance?
2. What data was used to train or fine-tune the model? Is it internal data, third-party data, or a combination?
3. How does the vendor test for bias, fairness, and accuracy? Can they provide the testing methodology and results?
4. Does the vendor monitor for model drift? If so, how often, and what triggers a re-validation?
5. What is the model’s error rate, and how is that defined and measured?
6. Can the vendor provide model documentation, including a description of the model architecture, inputs, and outputs?
7. Are protected-class proxies, such as zip code or credit data, used as inputs? If so, how are they managed under state insurance law?

Security and Privacy

8. Where is insurer data processed and stored? Is it in the United States, and does it cross borders?
9. Will the insurer’s data be used to train the vendor’s models or improve products for other customers?
10. What access controls and encryption standards are in place?
11. Does the vendor have a documented incident response plan? Has it been tested?
12. Does the vendor comply with relevant insurance data privacy requirements, including state-specific laws?

Governance and Compliance

13. Does the vendor have a written AI governance framework? Who is accountable for it?
14. Is the vendor familiar with the NAIC Model Bulletin and relevant state insurance AI requirements?
15. Can the vendor provide evidence of its own testing, validation, and monitoring practices?
16. Has the vendor been subject to any regulatory enforcement, litigation, or public complaints related to its AI systems?

Operations and Business Continuity

17. What is the service-level agreement for uptime, latency, and support?
18. How much notice will the vendor give before model updates, retraining, or architecture changes?
19. What happens to the insurer’s data if the contract is terminated? Is there a defined exit process?
20. Does the vendor have a business continuity and disaster recovery plan that covers AI operations?

If a vendor cannot answer a question, that is not automatically a disqualifier. But it is a gap that must be documented, remediated, or accepted at the right level of risk oversight.

Contract Clauses That Matter

The vendor questionnaire is only the first half of the control. The contract is where the control becomes enforceable. At minimum, a high-risk AI vendor contract should include the following:

• Audit rights. The insurer must be able to audit the vendor’s AI systems, data practices, and compliance controls, or to receive audit reports from a qualified third party.
• Regulatory cooperation. The vendor must cooperate with regulatory inquiries and examinations related to the AI system and must notify the insurer of any regulatory contact concerning the system.
• Data use restrictions. The contract should clearly prohibit the vendor from using the insurer’s data to train models for other customers or for general product improvement without explicit consent.
• Model change notification. The vendor must notify the insurer before material changes to the model, training data, or intended use case.
• Indemnification and liability. The vendor should indemnify the insurer for claims arising from the vendor’s AI system, subject to negotiation.
• Exit and data portability. Upon termination, the vendor must return or securely destroy the insurer’s data and provide reasonable transition assistance.

A vendor that refuses to accept these clauses is not necessarily malicious. It may simply be a vendor that has never operated in a regulated insurance environment. Either way, the refusal is a data point for your risk assessment.

After the Contract: Ongoing Monitoring

Due diligence at signing is not enough. AI systems change. Models are retrained. New data sources are added. Vendors are acquired. A vendor risk assessment program must include ongoing monitoring.

At least annually, re-assess each high-risk vendor against the same checklist. Track whether the vendor’s error rates, bias testing results, or data practices have changed. Monitor consumer complaints and adverse outcomes for patterns that might trace back to the vendor’s AI system. And keep an internal record of any material model changes or incidents.

A practical monitoring calendar for high-risk AI vendors should include:

• Quarterly: Review of error rates, complaints, and any known model changes.
• Semi-annually: Review of security or privacy incidents, including any involving the vendor’s other customers.
• Annually: Full re-assessment against the checklist and a contract compliance review.
• Ad hoc: Immediate review after any significant model change, data breach, regulatory action, or litigation involving the vendor.

If the vendor’s AI system supports adverse decisions, such as coverage denials or premium increases, the monitoring should be more frequent and should include human review of a sample of decisions. See our framework on AI in health insurance governance for how this works in a high-stakes line of business.

FAQ

Does the NAIC Model Bulletin apply to third-party AI vendors? Yes, but indirectly. The bulletin applies to insurers. It requires insurers to perform due diligence, maintain contract controls, and monitor third-party AI systems. The vendor is not directly regulated by the NAIC, but the insurer’s regulatory obligations extend to the vendor’s system.²

Can an insurer rely on a vendor’s SOC 2 report instead of doing its own AI due diligence? No. SOC 2 covers information security controls, not model fairness, accuracy, or insurance-specific regulatory compliance. A SOC 2 report may be a useful input, but it does not replace the NAIC-required AI due diligence.

What is the most commonly missed vendor risk? Data use. Many vendor contracts do not explicitly prohibit the vendor from using the insurer’s data to train models for other customers. This is often discovered only after a model output looks suspiciously familiar to the insurer’s own claims data.

Should low-risk AI vendors be exempted from all oversight? No. Low-risk vendors should still undergo basic security and privacy review, including data processing location, access controls, and incident response. The risk classification determines the depth of review, not whether there is review at all.

Red Flags

Some vendor behaviors should trigger immediate escalation. The most common are:

• The vendor refuses to provide model documentation or validation records.
• The vendor will not explain how training data was selected or cleaned.
• The contract does not include audit rights or regulatory cooperation.
• The vendor claims its model is a “black box” that cannot be tested for bias.
• The vendor updates the model without notifying the insurer.
• The vendor’s data security practices are vague or undocumented.
• The vendor is not familiar with insurance-specific AI regulation.

Each of these is a signal that the vendor may not be ready for a regulated insurance environment. The compliance cost of working with such a vendor often exceeds the purchase price.

Conclusion

AI vendor risk assessment is not a procurement exercise. It is a compliance exercise disguised as procurement. The NAIC Model Bulletin, NY DFS guidance, and Colorado’s regulation all say the same thing: the insurer owns the outcome, even when the algorithm is rented.

A good checklist helps you find the gaps before the examiner does. A good contract turns those gaps into enforceable obligations. And ongoing monitoring makes sure the vendor does not drift out of compliance while your business depends on it.

For a broader view of how vendor oversight fits into an insurance AI governance program, see our guide to NAIC AI governance requirements and our analysis of Colorado SB 26-189’s impact on insurers.

Footnotes

1. National Association of Insurance Commissioners, “Health Insurance Artificial Intelligence/Machine Learning Survey Results,” May 2025: https://content.naic.org/sites/default/files/inline-files/Health%20Survey%20Report%20-%20FINAL%205.9.25.pdf ↩

2. National Association of Insurance Commissioners, “Model Bulletin on the Use of Artificial Intelligence Systems by Insurers,” December 2023: https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf ↩ ↩² ↩³

3. Wilson Elser, “Artificial Intelligence Governance for Insurers,” January 2026: https://www.wilsonelser.com/publications/artificial-intelligence-governance-for-insurers ↩

4. New York State Department of Financial Services, “Insurance Circular Letter No. 7 (2024): Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing,” July 11, 2024: https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07 ↩

5. Colorado Division of Insurance, “Notice of Adoption - Amended Regulation 10-1-1 Governance and Risk Management Framework,” 2025: https://doi.colorado.gov/announcements/notice-of-adoption-amended-regulation-10-1-1-governance-and-risk-management-framework ↩