The defensibility standard for fraud investigation AI: what "AI you can defend" actually requires

"Defensible AI" is becoming a contested phrase in insurance, and a document-review vendor is currently the loudest voice defining it - around source-linked medical summaries that hold up in litigation. That bar is correct for document review and too low for fraud investigation. A defensible document summary needs observations linked to the underlying record. A defensible fraud finding needs that plus a complete evidence chain, a reconstructable decision trail, and explicit support for the denial or referral the carrier acts on.

The reason the gap matters is the downstream. A document summary is an internal artifact; a fraud finding becomes a denial, an SIU or SAR referral, or a paid claim - each of which is litigable and regulator-reviewable in a way a chronology is not. This post defines a three-tier defensibility standard, separating document review, flag-only detection, and full investigation, and argues that only an audit-trail-native, end-to-end investigation clears the highest bar.

It is written for Marcus, the SIU Director whose top fear is an AI conclusion he cannot defend in a deposition, an examination under oath, or a SAR filing, and for Lin, the Compliance Officer who owns the antifraud-plan filing and asks whether the output satisfies California 10 CCR 2698.36 and NAIC Model Act #680. For where this sits in the broader category, start with the guide to autonomous AI claims investigation.

The phrase doing the most work in insurance AI marketing right now is "defensible." Document-review vendors have built the cleanest version of it: AI whose output withstands legal scrutiny because every observation connects directly to the underlying record, the methodology is consistent across files, and the report is audit-ready. Wisedocs, a medical-record summarization vendor, has put this framing in front of the market through a sponsored piece in Claims Journal and a talk at the CLM Annual Conference. Their claimed gains are real for what they do: up to an 80% reduction in document-review time, roughly a 150% capacity increase, and chronology turnaround compressed from 14 days to 2.

Those are document-review numbers, and the bar behind them is the right bar for document review. The question a medical chronology has to answer is narrow: does this observation trace to a page in the record. Source-linked, consistent, audit-ready - that clears it. The problem is that "defensible AI in claims" is being treated as if it ends there, when document review is one phase of a fraud investigation, not the whole of it. From fraud detection to fraud resolution is the distinction the market is collapsing.

Look at where AI in claims actually sits today. Bain's 2025 Claims Maturity Assessment of 81 P&C insurers worldwide found that 78% use generative AI in some capacity, but only 4% have scaled it, and the common live use cases are summarizing lengthy documents, supporting customer communications, and detecting potential fraud, per Bain & Company. Summarization and detection. The layer that turns a summary and a flag into a resolved, defensible finding is the one almost no carrier has scaled - and it is the layer where the defensibility question is hardest, because that is where the carrier takes an action a court can review.

The reason the investigation bar is higher is the consequence of being wrong. A flawed document summary produces a re-read. A flawed fraud finding produces a denial that gets litigated, an SIU or SAR referral that names a person, or a paid claim that should not have been paid. Each of those is an action the carrier takes against a policyholder or a regulator looks at, and each is exactly the surface on which bad-faith and unfair-claims-practices exposure lives.

Start with the size of the pool the action sits on top of. Insurance fraud costs the US an estimated $308.6 billion a year, with $45 billion in property and casualty and $34 billion in workers compensation, per the Insurance Information Institute citing the Coalition Against Insurance Fraud. Roughly 10% of P&C claims involve some element of fraud. That is the input volume that has to clear the defensibility bar at the point of decision, not merely at the point of detection. The flag tells you to look; the finding is what you act on, and the finding is what gets tested.

The clearest illustration of the downside is an automated decision made without a defensible, individualized investigation trail. In one widely reported case, a jury returned a $200 million verdict against Sierra Health & Life - $40,000 in compensatory damages and $160 million in punitive - over an automated process that denied a treatment claim without considering the insurer's duty of good faith, per Matt Sharp Law. The case was a health insurer, but the principle is line-agnostic: an action taken without a reconstructable, individualized basis is what converts a routine denial into catastrophic exposure.

The systematic version of that principle is the Unfair Claims Settlement Practices Act, NAIC Model #900, adopted in some form across the states. It makes it an unfair practice to refuse to pay a claim without conducting a reasonable investigation, and to fail to adopt and implement reasonable standards for the prompt investigation of claims. A denial is not defensible because a document was summarized accurately. It is defensible because a reasonable investigation supported it and the carrier can reconstruct what that investigation found. That is the bar a fraud finding has to clear, and it is a different bar from the one a chronology clears.

Defensibility is not one bar; it is three. The amount of evidence, traceability, and decision support required rises as you move from organizing documents, to flagging suspicion, to resolving a claim. Treating all three as the same standard is how a tool that clears the document bar gets sold as if it clears the investigation bar.

Tier 1: Document review

The job is to organize a record and link each observation back to its source page. Document-review vendors do this well, and a source-linked, consistently formatted chronology is the right defensible artifact for the task. The same source-linking logic shows up in adjacent fintech document tools - Ocrolus and similar parsers in lending KYC - but the bar is identical: trace the observation to the record. What this tier does not produce is an investigation. It does not reach the signals that live outside the documents, and it does not support the action the carrier takes.

Tier 2: Flag-only detection

Detection vendors - FRISS, Shift Technology, Verisk - score a claim and surface it for review. That is genuinely useful and Hesper sits downstream of all three; detection is upstream, investigation is downstream. But a score is not a defensible finding. Rules-based detection runs a 60-85% false-positive rate, so the flag is noisy by construction: it tells an investigator to look, not what is true. The model features behind the score are rarely reconstructable as a per-decision trail, and the vendor hands the flag to a human rather than supporting the action taken. Detection defensibility means "we surfaced this for review." It is a real claim, and a lower one than the finding requires.

Tier 3: Full investigation

The job is to take the flag and produce the finding the carrier acts on. That requires evidence across every signal type, a reconstructable decision trail, and explicit support for the denial or referral. Hesper runs 15+ investigation phases in parallel on every flagged claim - document forensics, OSINT, statement cross-reference, timeline reconstruction, financial-pattern analysis - and logs each decision with its sources, reasoning, and timestamps, lifting coverage from the roughly 25% a manual team reaches to 100% of flagged claims. This is the only tier whose output is the artifact a carrier defends in a deposition or an examination. The table below maps the three tiers against the requirements that separate them.

Three components separate a defensible finding from a defensible document. Miss any one and the finding is contestable, regardless of how well-sourced the underlying summary is.

The first is a complete evidence chain across every signal the case touched. A claim is not only its documents. It is the recorded statements that may contradict the documents, the OSINT that places a claimant somewhere the claim says they were not, the timeline that does or does not hold together, and the financial patterns that connect this claim to others. A document summary covers one of those. A defensible finding has to assemble all of them into a single chain, because the question on examination is not "what did the medical records say" but "what did the investigation find, across everything available."

The second is a reconstructable decision trail. For each conclusion the investigation reaches, there has to be a record of the sources it drew on, the reasoning that connected them, and the timestamp at which it happened. This is what lets an SIU Director sit in a deposition and reconstruct, step by step, how the finding was reached - and what lets a compliance officer hand a state DOI examiner a chain they can read. Hesper logs every decision the agent makes with sources, reasoning, and timestamps for exactly this reason. The how-to companion to this is our walkthrough on how to generate an audit-ready fraud investigation report in under an hour.

The third is explicit support for the action the carrier takes. A defensible finding does not stop at organizing information; it supports the denial, the payment, or the SIU/SAR referral with the reasoning behind it. This is the component detection and document tools both leave to the human, and it is the one the Unfair Claims Settlement Practices Act puts the most weight on. The action is what gets litigated, so the action is what has to be supported.

Those three components are also what makes the output map cleanly to the carrier's compliance obligations. An evidence chain plus a decision trail plus action support is, in regulatory terms, a documented decision - which is what California 10 CCR 2698.36 requires of an SIU investigation and what an antifraud plan filed under NAIC Model Act #680 has to describe. The defensibility standard and the compliance standard are the same standard viewed from two angles.

The defensibility bar is not a vendor invention. It is already written into the regulatory frame, and AI does not lower it - AI has to meet it. The NAIC adopted the Model Bulletin on the Use of Artificial Intelligence Systems by Insurers on December 4, 2023, and roughly two dozen states have adopted it since. It explicitly contemplates AI deployed across "claim management" and "fraud detection," not only underwriting, which puts AI fraud decisions squarely inside the regulator's field of view.

The bulletin names the risks directly. AI "can present unique risks to consumers, including the potential for inaccuracy, unfair discrimination, data vulnerability, and lack of transparency and explainability." It asks insurers to assess models for "interpretability, repeatability, robustness, regular tuning, reproducibility, traceability, model drift, and the auditability of these measurements where appropriate." Traceability and auditability are the regulator's own words. A black-box score that cannot reconstruct its own reasoning is in tension with that expectation; an evidence chain logged per decision is built for it.

The bulletin also ties AI-driven actions back to the older model laws: an insurer's use of AI "must not violate" the Unfair Trade Practices Act (Model #880) or the Unfair Claims Settlement Practices Act (Model #900). And it lists the documentation a regulator may request during an investigation or examination of a specific AI system. The practical reading for a compliance officer is simple: when a state DOI pulls an AI-assisted case on audit, it will expect a documented, traceable, auditable basis for the decision. That is the same artifact the SIU Director needs for a deposition.

Defensibility therefore has a data-handling dimension as well as an evidentiary one. The audit trail has to be secure, the data residency has to be defensible, and the vendor has to sit cleanly inside the carrier's third-party-oversight obligations - which is why the security review and the defensibility review are the same conversation for Lin and Priya. We cover that side in detail in SOC 2 and data handling for AI fraud investigation.

When a vendor says its AI is defensible, the right response is to ask which tier they mean. "Source-linked" is necessary but not sufficient at the investigation layer, and the difference is buyable in five procurement questions Marcus and Lin can put to any vendor in writing.

1. Does the output include a reconstructable decision trail - sources, reasoning, and timestamps for each conclusion - or only a final score or summary.
2. Does the evidence chain cover non-document signals (OSINT, recorded statements, timeline reconstruction, financial patterns), or only the documents fed in.
3. Can an investigator see what the agent did, override it, and produce a documented trail for a state DOI examination.
4. Does the output map to the carrier's antifraud-plan filing under NAIC Model Act #680 and to documented-decision rules such as California 10 CCR 2698.36.
5. Does the vendor support the action the carrier takes - the denial or the SIU/SAR referral - or does it stop at organizing information and hand off to a human.

A document-review vendor will answer the first question for the summary and the others with a no, and that is the correct answer for their layer. A detection vendor will answer most of them with a no and a hand-off, which is correct for theirs. Those are not failures; they are the boundary of what each tier does. The point of the questions is to surface the boundary, so a carrier does not buy a document tool or a detection score expecting an investigation artifact.

This also clarifies how the tools fit together. A carrier runs detection and investigation at the same time - Hesper is complementary to FRISS, Shift Technology, and Verisk, not a replacement. Detection produces the flag; investigation produces the defensible finding from it. The investigator's role shifts from executing the 14+-day manual workflow to reviewing and deciding on findings produced in 2-4 hours, which is where human judgment actually belongs. Defensibility should be a line item in any procurement, and for the full rubric it sits inside, see our 12-point checklist for evaluating AI fraud investigation vendors.