---
title: "The chief risk officer's view of AI claims investigation: model risk, controls, and concentration"
description: "How the CRO evaluates AI claims investigation - as a model-risk and third-party-risk object, not on ROI. Why investigation-layer AI is a lower risk class, and the six controls that clear it."
date: "2026-07-14"
lastModified: "2026-11-16"
author: "Nitish Badu"
tags: ["Guides"]
canonical: "https://gethesperai.com/blog/chief-risk-officer-ai-claims-investigation/"
---

# The chief risk officer's view of AI claims investigation: model risk, controls, and concentration

> **TL;DR** A chief risk officer does not evaluate AI claims investigation on ROI - the deal survives only if the tool fits inside the carrier's existing control framework. Investigation-layer AI carries a lower model-risk class than pricing or underwriting AI because it produces evidence for a human SIU lead to adjudicate rather than making the coverage decision that NAIC, NYDFS, and Colorado frameworks target. An audit-trail-native agent is what lets that decision enter a model inventory at all.
>
> - The CRO holds veto power, not the budget chair
> - Investigation AI is a lower model-risk class than pricing AI
> - NAIC bulletin adopted in ~24 states by March 2025
> - Under-investigation is its own risk: UCSPA Model #900

- **$308B** - Annual US insurance fraud loss (The unmanaged-loss backdrop (Coalition Against Insurance Fraud))
- **~24 states** - Adopted the NAIC AI Model Bulletin (By March 2025 (state adoption trackers))
- **25% → 100%** - Flagged-claim coverage (Manual SIU vs Hesper (Hesper benchmark))
- **14+ days → minutes** - Investigation cycle time (Manual SIU vs Hesper (Hesper benchmark))

Every other seat at the table asks whether an AI investigation tool creates value. The chief risk officer asks a different question: does this tool create un-owned residual risk, and can it sit inside the model-risk, third-party-risk, and control framework before it goes live?

The CRO rarely holds the budget chair and rarely champions the purchase. What the CRO holds is veto power. A tool that cannot be inventoried, validated, and challenged does not get approved, no matter how strong the loss-ratio case.

The core argument: investigation-layer AI is a lower model-risk class than detection or pricing AI, because it produces evidence for a human adjudicator rather than making the pricing or coverage decision. Where that layer sits in the claims lifecycle is mapped in [the 2026 claims automation guide](/blog/claims-automation-pillar/). For the compliance officer who files the antifraud plan, see [the compliance officer's AI investigation deployment guide](/blog/compliance-officer-ai-investigation-deployment/).

## How a CRO classifies AI claims investigation

A chief risk officer classifies AI claims investigation as a model-risk and third-party-risk object. What matters is which risk categories the tool touches, which controls already govern them, and whether the residual risk is one the enterprise is willing to own.

A tool can clear loss-ratio, payback, and workflow review and still fail the risk review if it produces decisions no one can reconstruct.

| Buying seat | What they optimize for | What they say no to |
| --- | --- | --- |
| Claims VP | Loss-ratio movement in basis points | Regulatory exposure or adjuster-workflow disruption |
| CFO / Finance | Cost per case and payback period | ROI cases led with FTE displacement |
| SIU Director | Defensible, throughput-lifting workflow | Tools that bypass investigators or can't be defended |
| CIO / Technology | Integration cost and SOC 2 posture | Pre-SOC-2 vendors, training-on-customer-data clauses |
| Chief Risk Officer | Residual enterprise risk after controls | Un-owned model risk and un-exitable vendor dependency |

The exposure runs in both directions. Per the [Coalition Against Insurance Fraud](https://insurancefraud.org/fraud-stats/), insurance fraud steals at least $308 billion every year from American consumers. A CRO who treats only the AI tool as the risk, and not the fraud the carrier is failing to investigate, is pricing one side of the ledger. For where the veto seats sit, see [how to map the buying center for AI investigation](/blog/buying-center-mapping-ai-investigation/).

Hesper does not add a new scoring or decisioning model. It adds an evidence-production and documentation layer, and a human adjudicates. Every agent decision is logged with its sources, reasoning, and timestamps. That audit trail is the property that lets an AI investigation decision enter a control framework at all.

**Figure 1 (Risk classification) - An evidence producer sits in a lower risk class than a pricing model.** Same control framework, different class: what the output is decides how much model risk the tool carries.

|  | What the output is | Who makes the call | Where scrutiny concentrates |
| --- | --- | --- | --- |
| Pricing model (Underwriting, eligibility) | Who gets coverage, at what cost | The model's output is the decision | NYDFS CL-7, Colorado SB21-169 |
| Investigation agent (Downstream of detection) | Evidence and a reviewable report | A human SIU lead adjudicates | NAIC bulletin: audit trail, vendor oversight |

**Not exempt:** A lower class still needs an inventory entry, validation, effective challenge and documented governance.

**Takeaway:** The model's output is an input to a human decision, not the decision itself.

Source: Classification per this article's reading of SR 11-7 (Federal Reserve and OCC, April 2011; banking guidance used as vocabulary, not binding on insurers), the NAIC Model Bulletin (Dec 2023), NYDFS Circular Letter No. 7 (2024) and Colorado SB21-169.

## The model-risk lens: SR 11-7 applied

Model risk is the risk of adverse consequences from decisions based on incorrect or misused model output. A CRO applies four controls to any model: a model inventory entry, independent validation, effective challenge, and documented governance. AI claims investigation needs all four, but its risk class is lower than a pricing or underwriting model.

The vocabulary comes from banking: the Federal Reserve and OCC's [SR 11-7, "Supervisory Guidance on Model Risk Management"](https://www.federalreserve.gov/boarddocs/srletters/2011/sr1107.htm) (April 2011, since revised). SR 11-7 is banking guidance, not binding on insurers, so a CRO uses it as the vocabulary source rather than the rulebook.

### Why an investigation agent is a lower risk class

A pricing or underwriting model makes a decision about who gets coverage and at what cost. An investigation agent makes no such decision. It runs 15+ evidence-gathering phases in parallel on each flagged claim and outputs a reviewable report. A human SIU lead adjudicates.

That design is effective challenge by construction. In the SR 11-7 sense, effective challenge is the critical review of a model's output by informed, independent parties who can act on findings. When an SIU lead reviews every investigation package before any action, and can override the agent's reasoning, the control is the operating model, not a bolt-on.

*Figure: The low-tech option is not the low-risk option: the four controls a CRO stacks on any model - inventory, validation, effective challenge, documented governance - and where an investigation agent's audit trail clears each.*

## The regulatory perimeter: NAIC, NYDFS, Colorado

The AI-governance perimeter for US insurers is set by three frameworks: the NAIC Model Bulletin (adopted December 2023), NYDFS Insurance Circular Letter No. 7 (2024), and Colorado SB21-169. All three center on governance, third-party oversight, and testing for unfair discrimination. Their scrutiny concentrates on AI that influences underwriting, pricing, and eligibility.

**Figure 2 (The AI-governance perimeter) - Three frameworks, and all three aim hardest at pricing and eligibility.** Each arrived on its own track; claims investigation sits downstream of the decisions they target.

1. **Colorado: July 6, 2021** - SB21-169 signed: test data, algorithms and models for unfair discrimination
2. **NAIC: Dec 2023** - Model Bulletin on AI Systems: a documented AI Systems Program
3. **NYDFS: July 11, 2024** - Circular Letter No. 7 finalized: underwriting and pricing
4. **24th state: March 2025** - Wisconsin adopts the NAIC bulletin; Alaska was first, in Feb 2024

**Scope: Where the scrutiny concentrates.** Underwriting, pricing and eligibility: decisions about who gets coverage and at what cost. A claims investigation agent works on already-filed claims and produces evidence.

**Claims side:** Documented-decision anchors: California 10 CCR 2698.36 and NAIC Model Act 680, adopted in 48 states.

Sources: Colorado SB21-169 (signed July 6, 2021); NAIC Model Bulletin on the Use of AI Systems by Insurers (Dec 2023); NYDFS Insurance Circular Letter No. 7 (finalized July 11, 2024); state adoption trackers (Alaska first, Feb 2024; Wisconsin 24th, March 2025).

The [NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers](https://content.naic.org/sites/default/files/inline-files/AI%20Model%20Bulletin%20-%20April%202024.pdf) requires a documented AI Systems Program: governance, model validation and testing, third-party vendor oversight, and production of that documentation in market-conduct exams. Per state adoption trackers, roughly 24 states had adopted it or substantially similar guidance by early 2025. It is the broadest of the three, applying across AI uses, not only pricing.

NYDFS [Insurance Circular Letter No. 7](https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07), finalized July 11, 2024, requires model risk management with "independent review and effective challenge," and a three-step disproportionate-adverse-effect test. CL-7 governs underwriting and pricing. It is not a claims-investigation rule, but a CRO still wants its effective-challenge and vendor-responsibility language satisfied downstream.

[Colorado SB21-169](https://doi.colorado.gov/for-consumers/sb21-169-protecting-consumers-from-unfair-discrimination-in-insurance-practices) (signed July 6, 2021) requires insurers to test external consumer data, algorithms, and predictive models for unfair discrimination against protected classes and take corrective action. It is the model-inventory and vendor-oversight precedent now spreading across lines.

| Framework | Scope (which AI use) | What it requires | Applies squarely to investigation AI? |
| --- | --- | --- | --- |
| NAIC Model Bulletin (Dec 2023) | All AI use by insurers | Documented AIS Program: governance, validation, third-party oversight, exam-ready docs | Broadly, yes - satisfied via audit trail + vendor oversight |
| NYDFS CL-7 (2024) | Underwriting and pricing | Fairness, governance, transparency; effective challenge; disparate-impact testing | Governs pricing, not investigation; effective-challenge posture still wanted |
| Colorado SB21-169 / Reg 10-1-1 | ECDIS, algorithms, predictive models (pricing / eligibility) | Model inventory, data evaluation, unfair-discrimination testing, vendor oversight | Governs pricing / eligibility; inventory + vendor-oversight precedent applies |

> **The distinction a CRO should not blur**
>
> NYDFS CL-7 and Colorado SB21-169 govern underwriting and pricing. They do not directly govern claims investigation. An investigation agent operates downstream, on already-filed claims, and produces evidence rather than a coverage or pricing decision. Treating it as if it carried the full pricing-model burden over-classifies the risk; treating it as ungoverned under-classifies it. The right read is a lower class, still inside the control framework.

On the claims side, an audit-trail-native output maps to California 10 CCR 2698.36's documented-decision requirement and to the antifraud-plan filing obligations under NAIC Model Act 680, adopted in 48 states. When a market-conduct examiner pulls a Hesper-investigated case, the reconstructable record is already there. The evidence-chain substance is covered in [the AI fraud investigation defensibility standard](/blog/fraud-investigation-ai-defensibility-standard/).

## Concentration and third-party vendor risk

Concentration risk is the exposure created when a single vendor, model, or data source becomes a load-bearing dependency the carrier cannot quickly replace. Both the NAIC Model Bulletin and NYDFS CL-7 are explicit that the insurer retains full responsibility for third-party tools and may not rely solely on a vendor's claim.

So a CRO asks whether the tool creates a single point of failure. The mitigants are structurally stronger than for a decisioning model:

- It is complementary to the detection stack - FRISS, Shift Technology, and Verisk - not a replacement, and it can also run standalone.
- The output is a portable, human-readable evidence record, not an opaque score.
- Standard vendor-risk controls apply cleanly: audit rights, exit and data-portability terms, SOC 2, and no training on the carrier's data.

| Concentration-risk question | Weak-mitigation answer | Investigation-AI answer (Hesper) |
| --- | --- | --- |
| Is this a load-bearing dependency? | Replaces the detection stack | Complementary and standalone-capable - adds a layer, swaps nothing |
| Can the output survive vendor exit? | Opaque score, not reconstructable | Portable, human-readable evidence record |
| Who owns the risk of the tool? | Vendor claims transfer it | Carrier retains it - controls priced accordingly (audit rights, SOC 2) |
| Is the carrier's data exposed? | Ambiguous data-flow / training clauses | No training on customer data; documented data flows |

Because Hesper sits downstream of detection rather than replacing it, the detection dependency the carrier already governs stays where it is. The CRO is adding a documented layer, not concentrating more decisioning into fewer vendors. The CIO's side of that review is in the [CIO checklist for AI investigation rollout](/blog/cio-checklist-ai-investigation-rollout/).

## The under-investigation risk you may be mispricing

Under-investigation is a regulatory and operational risk in its own right. The NAIC Unfair Claims Settlement Practices Act (Model #900) prohibits, among 14 enumerated practices, "failing to adopt and implement reasonable standards for the prompt investigation of claims" and "refusing to pay claims without conducting a reasonable investigation."

Most CROs under-weight this risk because the status quo presents as normal operations. Manual SIU teams typically investigate only about 25% of flagged claims; the rest are paid, denied without full work, or queued indefinitely. That 75% is an accepted but unpriced risk sitting in two places at once: the loss-leakage tail and the [Unfair Claims Settlement Practices Act (Model #900)](https://content.naic.org/sites/default/files/model-law-900.pdf) exposure.

**Figure 3 (Under-investigation risk) - The 75% of flags nobody fully works is a risk sitting in two places.** Manual capacity sets the coverage ceiling, and what falls below it lands in the leakage tail and the unfair-claims tail at once.

100% minus 25% equals 75%

- Flagged claims: 100%
- Fully investigated, manual SIU: about 25% (Hesper internal benchmark)
- Paid, denied unworked, or queued: 75% (Accepted, but unpriced)

**Flagged: Two tails, one cause.**

- Leakage Fraud paid on flagged claims that were never fully investigated
- Model #900 One of 14 practices: refusing to pay claims without a reasonable investigation

**Takeaway:** Under-investigation is a risk the control framework should be pricing, not normal operations.

Sources: NAIC Unfair Claims Settlement Practices Act (Model #900), 14 enumerated practices. Manual coverage of flagged claims: Hesper internal benchmark, not an industry census.

Seen that way, AI investigation is risk reduction, not only efficiency. Moving flagged-claim coverage from roughly 25% to 100% narrows both tails. A manual SIU investigation runs 14+ days per case, and that per-case cost is what sets the 25% ceiling. An agent works a flagged claim in minutes, so throughput is no longer capped by headcount.

| Flagged-claim coverage: manual SIU vs Hesper (Hesper internal benchmark) | Value | Share |
| --- | --- | --- |
| Manual SIU coverage | ~25% | 25% |
| Hesper coverage | 100% | 100% |

## The CRO's control checklist for approval

A CRO can approve AI claims investigation once six controls are in place. Each maps to a property of the tool rather than a promise.

1. Model-inventory entry - the agent is registered as a model with a defined owner, purpose, inputs, and outputs, at the lower, evidence-producing risk tier.
2. Independent validation - outputs can be independently reviewed and reproduced, because every phase is logged with sources, reasoning, and timestamps.
3. Audit trail - the output is reconstructable end to end, satisfying California 10 CCR 2698.36 and NAIC Model Act 680 documentation expectations.
4. Human-in-the-loop adjudication - an SIU lead reviews and can override every investigation package before any action.
5. Third-party and concentration mitigation - audit rights, exit and data-portability terms, SOC 2 posture, and no training on customer data.
6. Documented regulatory mapping - the tool's scope is mapped against the NAIC bulletin, CL-7, SB21-169, and UCSPA Model #900.

| Control | What the CRO needs to see | How investigation AI satisfies it |
| --- | --- | --- |
| Model inventory | Registered model, owner, lower risk tier | Evidence-producing agent, not a decisioning model |
| Independent validation | Reproducible, reviewable output | 15+ phases each logged with sources and reasoning |
| Audit trail | Reconstructable decision record | Audit-trail-native (CA 10 CCR 2698.36, NAIC 680) |
| Effective challenge | Human review with override | SIU lead adjudicates every package |
| Concentration mitigation | Exitable, portable, non-exclusive | Complementary + standalone; portable evidence record |
| Regulatory mapping | Scope mapped to each framework | Lower class vs pricing AI; UCSPA under-investigation covered |

Cleared against those six, the enterprise-risk veto becomes an approval. The risk is bidirectional: the tool must fit the control framework, and the status quo of 25% coverage is itself a risk the framework should be pricing. For discovery, privilege, and admissibility of AI-produced evidence, see [the general counsel's view of AI fraud investigation and litigation](/blog/general-counsel-ai-fraud-investigation-litigation/).

## Key takeaways

- The CRO evaluates AI claims investigation as a model-risk and third-party-risk object, not on ROI, and holds the veto.
- Investigation-layer AI is a lower model-risk class than pricing or underwriting AI because it produces evidence for a human SIU lead to adjudicate.
- The NAIC Model Bulletin, NYDFS Circular Letter No. 7, and Colorado SB21-169 center on governance, third-party oversight, and unfair-discrimination testing, and concentrate on underwriting and pricing.
- Under-investigation is itself a risk: UCSPA Model #900 prohibits refusing to pay a claim without a reasonable investigation, so moving coverage from about 25% to 100% reduces the CRO's risk.
- An audit trail - every decision logged with sources, reasoning, and timestamps - is what lets an AI investigation decision enter a model inventory.

## Frequently asked questions

### How does a chief risk officer evaluate an AI system used in insurance claims?

A chief risk officer evaluates it as a model-risk and third-party-risk object, not on return on investment. The CRO asks four questions: does the system enter the model inventory, can it be independently validated, is there effective challenge through human review, and does it create vendor dependency the carrier cannot exit.

### Does AI claims investigation create model risk under SR 11-7 or NAIC guidance?

Yes, but at a lower risk class than pricing or underwriting AI. The key mitigant is structural: the agent runs evidence-gathering phases and outputs a reviewable report, and a human SIU lead makes the call. When every phase is logged with sources and reasoning, the output is auditable and reproducible.

### What is concentration risk with an AI vendor and how is it managed?

Concentration risk is the exposure created when a single vendor, model, or data source becomes a load-bearing dependency the carrier cannot quickly replace. A CRO manages it with contractual audit rights, documented exit and data-portability terms, verified security posture such as SOC 2, and confirmation the vendor does not train on the carrier's data.

### Is failing to investigate a flagged claim a compliance risk?

Yes. The NAIC Unfair Claims Settlement Practices Act (Model #900) enumerates 14 prohibited practices, including refusing to pay a claim without conducting a reasonable investigation. A carrier that pays or denies flagged claims without adequate investigation carries regulatory exposure on top of the loss leakage from paying fraud.

### How does Hesper AI's audit trail support model governance and regulatory exams?

Every decision the investigation agent makes is logged with its sources, reasoning, and timestamps, across 15+ phases run per claim. A human SIU lead validates the evidence package before any action. The trail maps to California 10 CCR 2698.36's documented-decision requirement and the antifraud-plan filing obligations under NAIC Model Act 680.
