---
title: "The chief risk officer's view of AI claims investigation: model risk, controls, and concentration"
description: "How the CRO evaluates AI claims investigation - as a model-risk and third-party-risk object, not on ROI. Why investigation-layer AI is a lower risk class, and the six controls that clear it."
date: "2026-07-14"
lastModified: "2026-07-14"
author: "Nitish Badu"
tags: ["Guides"]
canonical: "https://gethesperai.com/blog/chief-risk-officer-ai-claims-investigation/"
---

# The chief risk officer's view of AI claims investigation: model risk, controls, and concentration

> **TL;DR** A chief risk officer does not evaluate AI claims investigation on ROI - the deal survives only if the tool fits inside the carrier's existing control framework. Investigation-layer AI carries a lower model-risk class than pricing or underwriting AI because it produces evidence for a human SIU lead to adjudicate rather than making the coverage decision that NAIC, NYDFS, and Colorado frameworks target. An audit-trail-native agent is what lets that decision enter a model inventory at all.
>
> - The CRO holds veto power, not the budget chair
> - Investigation AI is a lower model-risk class than pricing AI
> - NAIC bulletin adopted in ~24 states by March 2025
> - Under-investigation is its own risk: UCSPA Model #900

- **$308B** - Annual US insurance fraud loss (The unmanaged-loss backdrop (Coalition Against Insurance Fraud))
- **~24 states** - Adopted the NAIC AI Model Bulletin (By March 2025 (state adoption trackers))
- **25% → 100%** - Flagged-claim coverage (Manual SIU vs Hesper (Hesper benchmark))
- **14+ days → 2-4 hrs** - Investigation cycle time (Manual SIU vs Hesper (Hesper benchmark))

A chief risk officer does not evaluate AI claims investigation on return on investment - the deal survives only if the tool fits inside the carrier's existing control framework. Every other seat at the table asks whether the tool creates value. The claims VP wants loss-ratio movement, the CFO wants payback, the SIU director wants a defensible workflow. The CRO asks a different question entirely: does this tool create un-owned residual risk, and can I get it inside my model-risk, third-party-risk, and control framework before it goes live.

That reframe changes how the whole procurement reads. The CRO rarely holds the budget chair and rarely champions the purchase. What the CRO holds is veto power through the enterprise-risk lens. A tool that cannot be inventoried, validated, and challenged does not get approved no matter how strong the loss-ratio case. This post walks the CRO's actual review: how the tool gets classified, the model-risk vocabulary a CRO borrows from banking, the regulatory perimeter for insurer AI, concentration and vendor risk, and the under-investigation exposure most risk functions are mispricing.

The core insight, stated up front: investigation-layer AI is a lower model-risk class than detection or pricing AI, because it produces evidence for a human adjudicator rather than making the pricing or coverage decision that the NAIC bulletin, NYDFS Circular Letter No. 7, and Colorado SB21-169 are aimed at. That distinction is what this post spends its length defending. For the seat next door - the compliance officer who files the antifraud plan - see [the compliance officer's AI investigation deployment guide](/blog/compliance-officer-ai-investigation-deployment).

## How a CRO classifies AI claims investigation

A chief risk officer classifies AI claims investigation as a model-risk and third-party-risk object that must fit an existing control framework before it can be approved. The classification is not about whether the tool works. It is about which risk categories the tool touches, which controls already govern those categories, and whether the residual risk after those controls is one the enterprise is willing to own.

This is where the CRO's evaluation diverges from every other buyer. Each seat in the buying center optimizes for something different, and the CRO's variable is the one that can stop a deal cold. A tool can clear loss-ratio, payback, and workflow review and still fail the risk review if it produces decisions no one can reconstruct. That is why a CRO reads a vendor pitch backward from the failure modes rather than forward from the value story.

| Buying seat | What they optimize for | What they say no to |
| --- | --- | --- |
| Claims VP | Loss-ratio movement in basis points | Regulatory exposure or adjuster-workflow disruption |
| CFO / Finance | Cost per case and payback period | ROI cases led with FTE displacement |
| SIU Director | Defensible, throughput-lifting workflow | Tools that bypass investigators or can't be defended |
| CIO / Technology | Integration cost and SOC 2 posture | Pre-SOC-2 vendors, training-on-customer-data clauses |
| Chief Risk Officer | Residual enterprise risk after controls | Un-owned model risk and un-exitable vendor dependency |

The reframe matters because the backdrop is a live governance exposure in both directions. Per the [Coalition Against Insurance Fraud](https://insurancefraud.org/fraud-stats/), insurance fraud steals at least $308 billion every year from American consumers, and fraud is present in roughly 10% of P&C claims. A CRO who treats only the AI tool as the risk, and not the fraud the carrier is failing to investigate, is pricing one side of the ledger. The layered model - prevention, detection, investigation - matters here precisely because the risk profile differs by layer. To place this within the full committee, see [how to map the buying center for AI investigation](/blog/buying-center-mapping-ai-investigation), which lays out where the veto seats sit; this post is the deep dive on the seat that blocks.

The Hesper-specific classification point is structural. Hesper does not add a new scoring or decisioning model. It adds an evidence-production and documentation layer downstream of a human adjudication - from fraud detection to fraud resolution. Every agent decision is logged with its sources, reasoning, and timestamps. Audit-trail-native by design is not a marketing line for the CRO; it is the property that lets an AI investigation decision enter a control framework at all.

## The model-risk lens: SR 11-7 applied

Model risk is the risk of adverse consequences from decisions based on incorrect or misused model output. A CRO applies four controls to any model that enters the enterprise: a model inventory entry, independent validation, effective challenge, and documented governance. AI claims investigation belongs in the inventory and needs all four - but its risk class is lower than a pricing or underwriting model, and the reason is structural, not cosmetic.

The vocabulary here comes from banking. The Federal Reserve and OCC's [SR 11-7, "Supervisory Guidance on Model Risk Management"](https://www.federalreserve.gov/boarddocs/srletters/2011/sr1107.htm) (April 2011, since revised) defined the model inventory, independent model validation, and "effective challenge" concepts that insurance CROs now borrow. SR 11-7 is banking guidance, not binding on insurers, so a CRO uses it as the vocabulary source rather than the rulebook. But the four controls travel cleanly, and they are the frame through which an investigation agent gets read.

### Why an investigation agent is a lower risk class

The distinction that lowers the class: a pricing or underwriting model makes a decision about who gets coverage and at what cost. An investigation agent makes no such decision. It runs evidence-gathering phases - 15+ of them in parallel on each flagged claim - and outputs a reviewable report. A human SIU lead adjudicates. The model's output is an input to a human decision, not the decision itself.

That design maps onto effective challenge by construction. Effective challenge, in the SR 11-7 sense, is the critical review of a model's output by informed, independent parties who can act on findings. When an SIU lead reviews every investigation package before any action - and can override the agent's reasoning - the effective challenge is not a bolted-on control. It is the operating model. The investigator's role shifts from execution to decision-making, which is exactly the human-in-the-loop posture a CRO wants to see documented.

> The single property that lets an AI investigation decision enter a model inventory is reconstructability. If every phase is logged with its sources, reasoning, and timestamps, the output is auditable and reproducible - which is what independent validation and effective challenge require. Without that log, an investigation agent is a black box a CRO cannot sign off on.
>
> - Hesper AI product research

*Figure: The low-tech option is not the low-risk option: the four controls a CRO stacks on any model - inventory, validation, effective challenge, documented governance - and where an investigation agent's audit trail clears each.*

## The regulatory perimeter: NAIC, NYDFS, Colorado

The AI-governance perimeter for US insurers is set by three frameworks: the NAIC Model Bulletin (adopted December 2023, in roughly two dozen states since), NYDFS Insurance Circular Letter No. 7 (2024), and Colorado SB21-169 - all of which center on governance, third-party oversight, and testing for unfair discrimination. Their scrutiny concentrates on AI that influences underwriting, pricing, and eligibility. That is the strict end of the spectrum, and it is where the contrast with investigation AI gets drawn.

The [NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers](https://content.naic.org/sites/default/files/inline-files/AI%20Model%20Bulletin%20-%20April%202024.pdf) requires a documented AI Systems Program: governance, risk management, and internal controls across the AI lifecycle, model validation and testing (including for unfair and proxy discrimination), third-party AI vendor and data oversight, and production of that documentation in market-conduct exams. Per state adoption trackers, roughly 24 states had adopted it or substantially similar guidance by early 2025, with Wisconsin the 24th in March 2025 and Alaska first in February 2024. This is the broadest of the three - it applies across AI uses, not only pricing.

NYDFS [Insurance Circular Letter No. 7](https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07), finalized July 11, 2024, sets three principles - fairness, governance and risk management, and transparency. It requires board and senior-management oversight, model risk management with "independent review and effective challenge," and a three-step disproportionate-adverse-effect test: identify any disparate impact, establish a legitimate rationale, then search for a less discriminatory alternative, all documented and available to regulators. Critically, CL-7 governs underwriting and pricing. It is the strict boundary case, not a claims-investigation rule - but its effective-challenge and vendor-responsibility language is exactly what a CRO still wants satisfied downstream.

[Colorado SB21-169](https://doi.colorado.gov/for-consumers/sb21-169-protecting-consumers-from-unfair-discrimination-in-insurance-practices) (signed July 6, 2021) requires insurers to test external consumer data, algorithms, and predictive models for unfair discrimination against protected classes and take corrective action. Colorado's Reg 10-1-1 (effective November 2023 for life insurers) operationalized it into a governance framework: catalogue and inventory models, evaluate data sources, remediate unfairly discriminatory outcomes, with explicit third-party vendor oversight duties. It is the model-inventory and vendor-oversight precedent now spreading across lines.

| Framework | Scope (which AI use) | What it requires | Applies squarely to investigation AI? |
| --- | --- | --- | --- |
| NAIC Model Bulletin (Dec 2023) | All AI use by insurers | Documented AIS Program: governance, validation, third-party oversight, exam-ready docs | Broadly, yes - satisfied via audit trail + vendor oversight |
| NYDFS CL-7 (2024) | Underwriting and pricing | Fairness, governance, transparency; effective challenge; disparate-impact testing | Governs pricing, not investigation; effective-challenge posture still wanted |
| Colorado SB21-169 / Reg 10-1-1 | ECDIS, algorithms, predictive models (pricing / eligibility) | Model inventory, data evaluation, unfair-discrimination testing, vendor oversight | Governs pricing / eligibility; inventory + vendor-oversight precedent applies |

> **The distinction a CRO should not blur**
>
> NYDFS CL-7 and Colorado SB21-169 govern underwriting and pricing - decisions about who gets coverage and at what cost. They do not directly govern claims investigation. A claims investigation agent operates downstream, on already-filed claims flagged by a separate detection layer, and it produces evidence rather than a coverage or pricing decision. Treating investigation AI as if it carried the full pricing-model burden over-classifies the risk; treating it as ungoverned under-classifies it. The right read is a lower class, still inside the control framework.

On the claims side, the documented-decision anchors are concrete. An audit-trail-native output maps to California 10 CCR 2698.36's documented-decision requirement and to the antifraud-plan filing obligations under NAIC Model Act 680, adopted in 48 states. When a market-conduct examiner pulls a Hesper-investigated case, the reconstructable record is already there. The evidence-chain substance behind that auditability is covered in [the AI fraud investigation defensibility standard](/blog/fraud-investigation-ai-defensibility-standard).

## Concentration and third-party vendor risk

Concentration risk is the exposure created when a single vendor, model, or data source becomes a load-bearing dependency the carrier cannot quickly replace. It is a distinct risk from model risk, and it is the second thing a CRO checks. Both the NAIC Model Bulletin and NYDFS CL-7 are explicit that the insurer retains full responsibility for third-party tools and may not rely solely on a vendor's claim. The vendor's assurances do not transfer the risk.

So a CRO asks directly whether an AI investigation tool creates a single point of failure. The mitigants for investigation-layer AI are structurally stronger than for a decisioning model. First, the tool is complementary rather than a replacement for the detection stack - complementary to FRISS, Shift Technology, and Verisk, not a replacement - so adopting it does not swap out an existing dependency and can also run standalone. Second, the output is a portable, human-readable evidence record, not an opaque score the carrier cannot reconstruct if the vendor relationship ends. Third, the standard vendor-risk controls apply cleanly: contractual audit rights, documented exit and data-portability terms, verified security posture such as SOC 2, and confirmation the vendor does not train on the carrier's data.

| Concentration-risk question | Weak-mitigation answer | Investigation-AI answer (Hesper) |
| --- | --- | --- |
| Is this a load-bearing dependency? | Replaces the detection stack | Complementary and standalone-capable - adds a layer, swaps nothing |
| Can the output survive vendor exit? | Opaque score, not reconstructable | Portable, human-readable evidence record |
| Who owns the risk of the tool? | Vendor claims transfer it | Carrier retains it - controls priced accordingly (audit rights, SOC 2) |
| Is the carrier's data exposed? | Ambiguous data-flow / training clauses | No training on customer data; documented data flows |

The concentration story is why complementary positioning matters to the CRO specifically, not just to procurement. Because Hesper sits downstream of detection rather than replacing it, the detection dependency the carrier already governs stays where it is. The CRO is adding a documented layer, not concentrating more decisioning into fewer vendors. The CIO reviews the same surface from the integration and SOC 2 side; that review is laid out in the [CIO checklist for AI investigation rollout](/blog/cio-checklist-ai-investigation-rollout).

## The under-investigation risk you may be mispricing

Under-investigation is a regulatory and operational risk in its own right: the NAIC Unfair Claims Settlement Practices Act (Model #900) prohibits, among 14 enumerated practices, "failing to adopt and implement reasonable standards for the prompt investigation of claims" and "refusing to pay claims without conducting a reasonable investigation." A carrier that pays or denies flagged claims without adequate investigation carries exposure on top of the direct loss leakage from paying fraud.

This is the risk most CROs under-weight, because the status quo does not present as a risk - it presents as normal operations. Manual SIU teams typically investigate only about 25% of flagged claims; the rest are paid, denied without full work, or queued indefinitely. That 75% is an accepted but unpriced risk sitting in two places at once: the loss-leakage tail and the [Unfair Claims Settlement Practices Act (Model #900)](https://content.naic.org/sites/default/files/model-law-900.pdf) exposure. A risk function focused only on AI model risk can miss that the low-tech option is not the low-risk option.

Reframed as the CRO would see it, AI investigation is risk reduction, not only efficiency. Moving flagged-claim coverage from roughly 25% to 100% narrows both tails. The cycle-time and cost figures make the shift feasible at scale: a manual SIU investigation runs 14+ days per case at roughly $2,500, against 2-4 hours at roughly $150 per case, and throughput moves from about 10 investigations per investigator per month to 800+. The residual-risk math changes when full, uniform investigation of every flag becomes operationally possible.

| Flagged-claim coverage: manual SIU vs Hesper (Hesper internal benchmark) | Value | Share |
| --- | --- | --- |
| Manual SIU coverage | ~25% | 25% |
| Hesper coverage | 100% | 100% |

There is a fairness dimension the CRO checks here too, and it points the same way. The disparate-impact scrutiny in CL-7 and SB21-169 is aimed at systems that decide coverage and price. A claims investigation agent operates downstream of those decisions. The residual concern a CRO tests is whether investigation is triggered and resourced consistently across claimants. Full, uniform coverage - the same 15+-phase investigation on every flag - is more consistent than selective manual investigation of roughly a quarter of flags. Consistency is itself a control.

## The CRO's control checklist for approval

A CRO can approve AI claims investigation once six controls are in place: a model-inventory entry, independent validation, an audit trail, human-in-the-loop adjudication, third-party and concentration mitigation, and documented regulatory mapping. The checklist turns the enterprise-risk veto into a set of pass conditions - and each maps to a property of the tool rather than a promise.

1. Model-inventory entry - the investigation agent is registered as a model with a defined owner, purpose, inputs, and outputs, classified at the appropriate (lower, evidence-producing) risk tier.
2. Independent validation - the agent's outputs can be independently reviewed and reproduced, because every phase is logged with sources, reasoning, and timestamps.
3. Audit trail - the output is reconstructable end to end, satisfying California 10 CCR 2698.36 and NAIC Model Act 680 documentation expectations and market-conduct exam production.
4. Human-in-the-loop adjudication - an SIU lead reviews and can override every investigation package before any action, which is effective challenge by construction.
5. Third-party and concentration mitigation - contractual audit rights, exit and data-portability terms, SOC 2 posture, no training on customer data, and a complementary (not replacement) footprint.
6. Documented regulatory mapping - the tool's scope is mapped against the NAIC bulletin, CL-7, SB21-169, and UCSPA Model #900, showing which apply squarely and how each is satisfied.

| Control | What the CRO needs to see | How investigation AI satisfies it |
| --- | --- | --- |
| Model inventory | Registered model, owner, lower risk tier | Evidence-producing agent, not a decisioning model |
| Independent validation | Reproducible, reviewable output | 15+ phases each logged with sources and reasoning |
| Audit trail | Reconstructable decision record | Audit-trail-native (CA 10 CCR 2698.36, NAIC 680) |
| Effective challenge | Human review with override | SIU lead adjudicates every package |
| Concentration mitigation | Exitable, portable, non-exclusive | Complementary + standalone; portable evidence record |
| Regulatory mapping | Scope mapped to each framework | Lower class vs pricing AI; UCSPA under-investigation covered |

Cleared against those six, the enterprise-risk veto becomes an approval. The point a CRO ends on is the same one that started this post: the risk is bidirectional. The tool must fit the control framework, and the status quo of 25% coverage is itself a risk the framework should be pricing. The legal-risk cousin of this analysis - discovery, privilege, and admissibility of AI-produced evidence - is covered in [the general counsel's view of AI fraud investigation and litigation](/blog/general-counsel-ai-fraud-investigation-litigation).

## Key takeaways

- The CRO evaluates AI claims investigation as a model-risk and third-party-risk object, not on ROI - the deal survives only if the tool fits inside the carrier's existing control framework, and the CRO holds veto power rather than the budget chair.
- Investigation-layer AI carries a lower model-risk class than pricing or underwriting AI because it produces evidence for a human SIU lead to adjudicate rather than making the adjudication itself, which is effective challenge by construction.
- The insurance AI-governance perimeter - NAIC Model Bulletin (Dec 2023, adopted in roughly half of states), NYDFS Circular Letter No. 7 (2024), and Colorado SB21-169 - centers on governance, third-party oversight, and testing for unfair discrimination, and concentrates on underwriting and pricing.
- Under-investigation is itself a regulatory and operational risk: UCSPA Model #900 prohibits refusing to pay a claim without a reasonable investigation, so moving flagged-claim coverage from about 25% to 100% reduces the CRO's risk, not just the SIU's workload.
- An audit-trail-native investigation agent - every decision logged with sources, reasoning, and timestamps - is what lets an AI investigation decision enter a model inventory and satisfy effective-challenge and documented-decision requirements.

## Frequently asked questions

### How does a chief risk officer evaluate an AI system used in insurance claims?

A chief risk officer evaluates an insurance AI system as a model-risk and third-party-risk object, not on its return on investment. The CRO asks four questions: does the system enter the model inventory, can it be independently validated, is there effective challenge through human review, and does it create concentration or vendor dependency the carrier cannot exit. For claims investigation AI specifically, the classification is usually lower risk than pricing or underwriting AI, because an investigation agent produces evidence and a documented recommendation that a human SIU lead adjudicates - it does not price, approve, or deny the claim. The tool is approvable once it fits the carrier's existing control framework and its decisions are logged with sources, reasoning, and timestamps.

### What regulations govern insurers' use of AI in the United States?

The core US perimeter is three frameworks. The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted December 2023 and adopted by roughly two dozen states since, requires a documented AI Systems Program covering governance, risk controls, model validation, and third-party oversight. NYDFS Insurance Circular Letter No. 7 (2024) governs AI in underwriting and pricing, setting principles of fairness, governance, and transparency and requiring testing for disproportionate adverse effects on protected classes. Colorado SB21-169 requires insurers to test data, algorithms, and predictive models for unfair discrimination and remediate harms. All three emphasize governance, third-party vendor oversight, and unfair-discrimination testing, and expect documentation available to regulators on request.

### Does AI claims investigation create model risk under SR 11-7 or NAIC guidance?

Yes, but at a lower risk class than pricing or underwriting AI. Model risk, in the SR 11-7 sense a CRO borrows from banking, is the risk of adverse consequences from decisions based on incorrect or misused model output. An investigation agent belongs in the model inventory and needs validation and effective challenge. The key mitigant is structural: an investigation agent does not make the adjudication. It runs evidence-gathering phases and outputs a reviewable report; a human SIU lead makes the call. That human-in-the-loop design is effective challenge by construction. When every phase is logged with sources and reasoning, the agent's output is auditable and reproducible, which is what model validation and NAIC's documentation expectations require.

### What is concentration risk with an AI vendor and how is it managed?

Concentration risk is the exposure created when a single vendor, model, or data source becomes a load-bearing dependency the carrier cannot quickly replace. Both the NAIC Model Bulletin and NYDFS Circular Letter No. 7 make clear the insurer retains full responsibility for third-party tools and may not rely solely on a vendor's claims. A CRO manages the risk with contractual audit rights, documented exit and data-portability terms, verified security posture such as SOC 2, and confirmation the vendor does not train on the carrier's data. For investigation AI, concentration is lower when the tool is complementary rather than a replacement for the detection stack and when its output is a portable, human-readable evidence record rather than an opaque score the carrier cannot reconstruct.

### Is failing to investigate a flagged claim a compliance risk?

Yes. The NAIC Unfair Claims Settlement Practices Act (Model #900) enumerates 14 prohibited practices, including failing to adopt reasonable standards for prompt investigation and refusing to pay a claim without conducting a reasonable investigation. So a carrier that pays or denies flagged claims without adequate investigation carries regulatory exposure, on top of the direct loss leakage from paying fraud. Because manual SIU teams typically investigate only about 25% of flagged claims, the under-investigation tail is an accepted but often unpriced risk. Moving coverage toward 100% of flagged claims narrows both the leakage tail and the unfair-claims-practices exposure, which is why a CRO can treat AI investigation as risk reduction rather than only efficiency.

### How does Hesper AI's audit trail support model governance and regulatory exams?

Hesper is audit-trail-native: every decision the investigation agent makes is logged with its sources, reasoning, and timestamps, across 15+ phases run per claim. For a CRO, that log is what lets an AI investigation decision enter a model inventory and satisfy the documentation and effective-challenge expectations in the NAIC Model Bulletin and state guidance. The output is a reviewable evidence package a human SIU lead validates before any action, which keeps adjudication with a person. On the claims side, the documented decision trail maps to California 10 CCR 2698.36's documented-decision requirement and the antifraud-plan filing obligations under NAIC Model Act 680. When a market-conduct examiner pulls a case, the reconstructable record is already there.

### Does AI claims investigation raise disparate impact or fair-lending-style concerns?

Less than pricing or underwriting AI, but a CRO still tests for it. The disparate-impact and adverse-action scrutiny in NYDFS Circular Letter No. 7 and Colorado SB21-169 is aimed primarily at systems that influence underwriting, pricing, or eligibility - decisions about who gets coverage and at what cost. A claims investigation agent operates downstream of those decisions, on already-filed claims that were flagged by a separate detection layer, and it produces evidence rather than a coverage or pricing decision. The residual concern a CRO checks is whether investigation is triggered and resourced consistently across claimants. Full, uniform coverage of flagged claims - the same 15+-phase investigation on every flag - is actually more consistent than selective manual investigation of roughly a quarter of flags.