Connected Claims USAFind us at booth 502, Signia by Hilton Orlando, Nov 3-4Booth 502 · Orlando, Nov 3-4Booth 502 · Connected ClaimsBook a meeting
Blog›Guides
GuidesOctober 1, 2026·13 min read·Nitish Badu, COO·Updated November 16, 2026

Claims decision evidence: what the claim-file rules asked for before AI

Drew Young names the gap: system evidence, governance evidence, decision evidence. US claim-file rules have required the third since 1978, and a decision log does not carry it.

NB
Nitish Badu · COO and Co-founder
October 1, 2026·13 min read·Updated November 16, 2026
The numbers behind this
418Violations cited from a 259-file claim sample in one market conduct examCalifornia Department of Insurance examination of the California FAIR Plan Association, review period January 2017 to March 2021, adopted 25 May 2022
38 vs 4Citations in that exam for a deficient investigation versus a deficient file10 CCR 2695.7(d) thorough, fair and objective investigation; 10 CCR 2695.3(a) file reconstruction
1978Year Pennsylvania's claim-file reconstruction rule took effect31 Pa. Code 146.3, adopted 15 December 1978, effective 16 December 1978
30US jurisdictions with the NAIC AI model bulletin adopted or their own AI rulesNAIC adoption map, status 31 August 2026: 26 adopted plus 4 with insurance-specific regulation or guidance

A decision log tells an examiner what your system decided. It does not tell them what the claim had been checked for first. Drew Young, founder of Veriscopic, named the gap in September 2026 by splitting AI accountability into three layers: system evidence, governance evidence and decision evidence. His question: "Why was the insurer entitled to rely on that decision when it was made?"

The third layer goes missing because it costs something. A cleared check produces nothing the adjuster needs and one more line to type, so under caseload pressure it is the first thing to stop being written down. Which decisions stay with a person at all is mapped in which claims decisions stay human.

Insurance decisions have long afterlives

Answer

What is decision evidence, and how is it different from an audit trail?

Decision evidence is the record that shows why an organisation was entitled to rely on a decision at the moment it was made. Drew Young separates it from system evidence, meaning what the model or workflow did, and governance evidence, meaning what controls and approvals were in place. An audit trail is system evidence.

The framework comes from a piece Young published in Insurance Edge on 22 September 2026. System evidence and governance evidence are what most AI assurance work produces. Decision evidence is what gets tested when somebody comes back.

01Three layers of evidence

Most AI assurance produces the first two layers. Examiners test the third.

Drew Young's split of AI accountability: what the system did, what controls existed, and why relying on the decision was justified.

  1. System evidenceWhat did the model, workflow or agent do? An audit trail lives here.
  2. Governance evidenceWhat controls, approvals, policies or oversight were in place?
  3. Decision evidenceWhy was the organisation entitled to rely on this decision, in this context?
NoteObservability of an agent's actions is still system evidence: an input to decision evidence, not a substitute for it.
1978The year Pennsylvania's claim-file reconstruction rule took effect. No US property and casualty carrier had to wait for AI to be asked for the third layer.
Source: Drew Young, founder of Veriscopic, in Insurance Edge, 22 September 2026 (framing credited, not endorsed; Hesper does not implement VES). Pennsylvania rule: 31 Pa. Code 146.3, effective 16 December 1978.

The companion piece he published on 1 September 2026 is the stronger one for claims. The evidence behind a decision may no longer exist together, "in the state in which the decision occurred", because it ends up "distributed across claims systems, underwriting platforms, emails, documents, model logs, data stores and people's memories."

For a head of claims operating under delegated authority, this is an annual event: the claims audit a capacity partner runs on a sample of files. The question in that audit is not what the decision was. It is what the file shows was looked at.

Six fields describe a decision. An investigation has more.

Answer

What should a claim decision record contain?

Five Sigma's published record carries six fields: the decision and timestamp, the inputs available, the actor with version, the delegated authority, the override path and the override history. Those are the right six for logging a decision. None of them records which checks ran on the claim and came back clean.

Five Sigma published the clearest public specification of a decision record on 17 September 2026. Its examiner framing is correct: "An examiner will ask you to reconstruct a specific decision on a specific claim on a specific date, and show who or what made it, on what basis, under whose authority, and who could have overridden it."

FieldFive Sigma's definitionWhat it establishes
Decision and timestampThe specific outcome, on the specific claim, at the specific momentThat a decision exists, and when it was taken
Inputs availableWhich documents, data fields, and third-party sources fed the decisionWhat was in front of the decision-maker, not what each source returned
ActorAdjuster, rule, or agent, named, with version and configurationWho or what acted, and in which build
AuthorityThe delegated authority level under which the decision was permittedThat the act fell inside its mandate
Override pathWho could have reversed it, and how the system surfaced that optionThat a human had a route in
Override historyWhether a human intervened, what changed, and the stated reasonWhether anybody took that route

The list is not wrong, and a carrier that can produce it on demand is ahead of most of the market. But the object is smaller than the thing being examined. What an examiner tests is the investigation that preceded the decision.

An approval trail tells us what happened. It doesn't necessarily preserve what made the decision make sense at that moment.

Drew Young, Insurance Edge, 1 September 2026

A property claim is denied on a late-reporting ground. Three years later the policyholder's counsel asks what else the carrier considered:

  • Was the prior-loss history pulled?
  • Were the other addresses associated with the insured run?
  • Was the weather data for the loss date checked against the reported cause?
  • Was the vacancy exclusion tested and found not to apply, or was it never read?

Every one of those is an activity inside the Model 902 definition of investigation, and not one of them is a field in the log.

That gap closes only when the artefact is produced by the work rather than written up after it. Hesper leaves behind the output of every investigation phase, including the ones that came back clean. Running the phases in parallel is what makes that affordable.

The claim-file rules already ask for the activities

Answer

Do US claim-file rules require documenting the investigation or just the decision?

The investigation. NAIC Model Regulation 902 requires documentation sufficient to permit reconstruction of the insurer's activities relative to each claim. California adds a second test on top of event reconstruction: the licensee's actions must be determinable from the file. New York and Pennsylvania use the same reconstruction construction.

Four texts run a reconstruction test on the claim file, and three of them are older than any claims AI on the market. None of them uses the phrase decision evidence. All of them describe it. Pennsylvania wrote its own version of the test in December 1978.

RuleWho it reachesThe reconstruction test, in the textRetention stated in the rule
NAIC Model Reg #902, Section 4.BModel text for unfair property and casualty claims settlement practicesDocumentation detailed enough to permit reconstruction of the insurer's activities relative to each claimAll open and closed files for the current year and the two preceding years (Section 4.A)
10 CCR 2695.3(a) (California)Every licensee handling California claimsDetail enough that pertinent events and the dates of the events can be reconstructed AND the licensee's actions pertaining to the claim can be determinedCurrent year plus the four preceding years
11 NYCRR 216.11 (Regulation 64)Insurers doing business in New YorkFiles maintained so that all events relating to a claim can be reconstructed by Department of Financial Services examiners; every communication datedNot stated in the section
31 Pa. Code 146.3Insurers subject to Pennsylvania claims rulesNotes and work papers in the detail that pertinent events and the dates of the events can be reconstructedNot stated; in force since 16 December 1978

NAIC Model Regulation 902 Section 4.B requires "detailed documentation" in each claim file "in order to permit reconstruction of the insurer's activities relative to each claim." Activities, not conclusions.

Section 3.G defines "Investigation" as "all activities of an insurer directly or indirectly related to the determination of liabilities under coverages afforded by an insurance policy or insurance contract." No distinction between activities that changed the outcome and activities that did not. A record organised around the outcome drops the larger subset.

Fifteen illustrative activities behind one liability determination. The three that moved the outcome are what a decision record keeps. The twelve that cleared are the negative space a reconstruction test asks about.

California wrote the second test explicitly. 10 CCR 2695.3(a) requires files "in such detail that pertinent events and the dates of the events can be reconstructed and the licensee's actions pertaining to the claim can be determined." A file can pass the first test and fail the second, which is the exact shape of the gap.

What a market conduct examiner actually opens

Answer

What does a market conduct examiner test on claim files?

A random sample of files, measured against standards one at a time, against a tolerance. The NAIC examination standards summary lists eleven claims standards. Two land on this subject: Standard 2, timely investigations are conducted, and Standard 5, claim files are adequately documented. They carry separate samples.

Examinations run on a tolerance rather than on perfection. A Kansas Insurance Department report of examination of the Liberty Mutual Group, covering the period to 31 December 2004, puts it plainly: "A tolerance standard of 7% is used for claim procedures and 10% is used for all other procedures." That report is twenty-two years old and tolerances vary by state, so read the figure as illustrative of the method.

The NAIC Market Regulation Handbook examination standards summary keeps the two standards apart, and that separation is what a decision log cannot survive. The log answers Standard 5 for one line item: a record exists, it is dated, it names the actor. Nothing in it describes an investigation.

02Market conduct standards

A decision log answers Standard 5. Standard 2 needs the investigation record.

Two of the eleven NAIC claims standards land on this subject, and examiners test them separately, each with its own sample.

Timely investigations (2)
Files documented (5)
Decision logOutcome, actor, authority
Timely investigations (2)Not addressed
Files documented (5)One line item: dated, actor named
Investigation record
Timely investigations (2)What was checked, cleared ones included
Files documented (5)Answered from the same artefact
7%Tolerance standard for claim procedures in one Kansas exam report, against 10% for all other procedures. Exams run on a tolerance, not on perfection; read the figure as the method, not a current rate.Period to 31 December 2004
Sources: NAIC Market Regulation Handbook, examination standards summary (Standard 2, timely investigations are conducted; Standard 5, claim files are adequately documented); Kansas Insurance Department report of examination of the Liberty Mutual Group, period to 31 December 2004. Tolerances vary by state.

The clearest itemised public example is the California Department of Insurance targeted examination of the California FAIR Plan Association, adopted on 25 May 2022. Examiners randomly selected 259 claim files and cited 418 violations.

California FAIR Plan Association exam: citations by category, from a 259-file random sample and 418 total violations (CDI report adopted 25 May 2022)

Failed to conduct a thorough, fair and objective investigation (10 CCR 2695.7(d))38 citations
Failed to document the justification for an adjustment (CIC 2051/2051.5, 10 CCR 2695.9(f))13 citations
Failed to maintain a reconstructable claim file (10 CCR 2695.3(a))4 citations

Thirty-eight citations for failing to "conduct and diligently pursue a thorough, fair and objective investigation". Four for failing to maintain a file detailed enough to reconstruct. The file did not fail on what was not written down. It failed on what was not checked.

Penalties attach. The Florida Office of Insurance Regulation's insurer compliance report for October 2025 lists seven property and casualty consent orders from market conduct examinations of hurricane claims handling, with fines from $50,000 to $400,000. They are not described as documentation findings, but they show these examinations carry money.

Carriers miss the investigation record because of throughput, not ignorance. A manual SIU investigation runs 14+ days per case against caseloads of 200+ cases per investigator, both Hesper internal benchmarks, so the negative findings go unwritten first. Why most flagged claims never get investigated is the same arithmetic from the capacity side.

The AI rulebook covers the model. The claim file covers the claim.

Answer

Does the NAIC AI model bulletin require insurers to document individual claim decisions?

Not directly. Section 4 of the bulletin lists what a department may request: the written AI Systems Program, model inventories, validation and drift documentation, and third-party contracts. Those are program-level and model-level records. The claim-level obligation comes from the unfair claims settlement practices rules in each state.

The NAIC adoption map, status as of 31 August 2026, records 26 jurisdictions that have adopted the AI model bulletin, plus four more with insurance-specific AI regulation or guidance of their own. A programme built against that surface is worth having. It is not the source of the claim-level obligation.

The NAIC's own issue brief of March 2026 is blunt: AI "does not alter insurers' legal obligations. Existing state insurance laws apply regardless of whether decisions are made by humans, algorithms, or third-party vendors." A carrier can satisfy Section 4 and still hold no decision evidence at claim level.

The one instrument drafted at claim level did not pass. An NCOIL model act, summarised in the minutes of the NAIC Big Data and AI Working Group meeting of 29 September 2025, would have barred AI as the sole basis for claim denials or adjustments. NCOIL paused it in March 2026. New Jersey's A5494 is the live state version, and it regulates who signs rather than what is in the file.

Negative findings are evidence, and audit practice says so

Answer

Does a claim file have to document what was ruled out?

Yes, if the file is meant to show the investigation happened. Audit practice is explicit about the equivalent duty: PCAOB Auditing Standard No. 3 requires documenting the procedures performed, identifying the items inspected, and retaining information that contradicts the final conclusion. Missing documentation casts doubt that the work was done.

Auditing wrote down what a defensible file looks like. PCAOB Auditing Standard No. 3, now AS 1215, requires that the auditor "document the procedures performed, evidence obtained, and conclusions reached" and that the documentation "clearly demonstrate that the work was in fact performed".

Paragraph 8 goes further than anything in the claims rulebook. Documentation must include "information the auditor has identified relating to significant findings or issues that is inconsistent with or contradicts the auditor's final conclusions." Contradictory evidence stays in the file by rule. Set the two records against the questions asked later.

The question asked laterA decision logAn investigation record
What was decided, and whenYes, by designYes
Who or what decided, under what authorityYesYes
Which inputs were available to the decisionYes, as a list of sourcesYes, plus what each source returned
Which checks ran and found nothingNot a field in the recordLogged with the same timestamp discipline as the findings
Which coverage exclusions were tested and did not applyNot a fieldEach tested exclusion, with the reason it was ruled out
Whether evidence inconsistent with the conclusion was retainedNot a fieldRetained, which is what PCAOB AS No. 3 paragraph 8 demands of auditors
Whether the investigation was thoroughNot addressedShown by the identification of items inspected
Which NAIC claims standard it answersStandard 5, claim files are adequately documentedStandard 2 and Standard 5, from one artefact

The practical difference is production cost. A decision log is cheap because it writes down an event that already happened. An investigation record is expensive at human throughput, because every cleared check is another minute of typing that produces no benefit to the adjuster doing it.

03Production cost

At human throughput, the cleared checks are the first thing left unwritten

Why the industry ended up with decision logs: a capacity problem wearing a documentation costume.

How the record compresses
  1. 1.A cleared check gives the adjuster nothing, and costs one more line to type
  2. 2.Under caseload pressure, it stops being written down
  3. 3.The decision survives in the file. The search that justified it does not.
Manual baselineHesper internal benchmark
14+ days per case
200+ cases per investigator
~25% of flagged claims get a full workup
Checks as agent work
minutes, not weeks
Cleared checks logged like findings
100% of flagged claims

A cleared check costs the same to preserve as one that hits.

Hesper internal benchmarks, not industry figures: 14+ days per manual SIU case, caseloads of 200+ cases per investigator, coverage of flagged claims from 25% to 100%.

A decision log records the checks that changed the answer. Defensibility lives in the ones that did not.

Hesper AI product research

Running the investigation as software changes that arithmetic. When the checks execute as agent actions, the record of each is a by-product rather than an extra task. Coverage of flagged claims moves from 25% to 100%, both Hesper internal benchmarks, and the file arrives in minutes rather than weeks.

What that file has to contain is specified in the defensibility standard for investigation AI, with the architecture underneath it in the guide to autonomous AI claims investigation.

The file you hand a regulator, a reinsurer, or a jury

Answer

Who reads a claim file after the claim closes?

Three audiences with one test between them. A market conduct examiner asks whether the investigation was conducted and documented. A capacity partner's claims audit asks what the delegated decision rested on. A court asks whether the record is trustworthy enough to admit. All three are reconstruction questions.

Admissibility is the strictest version of Young's question. Federal Rule of Evidence 803(6) admits a business record made at or near the time by someone with knowledge, kept in the course of a regularly conducted activity, where making the record "was a regular practice of that activity", unless "the method or circumstances of preparation indicate a lack of trustworthiness". Every clause is about process discipline, not about being right.

A file assembled by hand under deadline pressure and one assembled by an agent that logs every tool call face the same test, and only one carries the timestamps by construction. More in what survives a courtroom.

The capacity-partner version is the one that catches MGAs. A delegated-authority audit runs on a sample of files a year or more after the decisions in them, by someone with no access to the people who made them. A file of conclusions reads as a file of assertions.

With Hesper, agents take a claim from first notice to final recovery, and the evidence is a by-product of doing the work. The same discipline runs the whole path, mapped in the guide to automating the claims lifecycle:

  • Coverage analysis produces a cited position naming the endorsements it tested and rejected.
  • The investigation writes down every phase, including the ones that cleared.
  • Settlement valuation carries the comparables it used and the ones it discarded.
  • The recovery screen records the files where no recoverable third party was found.

This describes obligations, not compliance

Nothing here asserts that Hesper or any other system satisfies NAIC Model Regulation 902, 10 CCR 2695.3, 11 NYCRR 216.11, 31 Pa. Code 146.3 or any state's unfair claims settlement practices act. Those obligations sit on the insurer, and whether a given file meets them is a judgement for the carrier's own counsel.

Nobody decided that cleared checks should go unrecorded. Sampling-era investigation depth was a capacity constraint, not a policy choice. When the investigation runs on every file, the record of what was ruled out becomes a by-product of the work. That makes Young's third layer buildable at claim level, and the shortest route to decision evidence is producing the investigation that generates it.

Key takeaways

  • Decision evidence, in Drew Young's three-layer framing, is the record of why an insurer was entitled to rely on a decision when it was made.
  • The obligation is not new: NAIC Model Regulation 902 Section 4.B requires documentation that permits reconstruction of the insurer's activities, and Pennsylvania has run a version of the test since December 1978.
  • Five Sigma's six decision-record fields are the right six for logging a decision, and none of them carries the checks that came back clean.
  • Market conduct exams test timely investigations and file documentation as two separate standards, so a decision log answers one while an investigation record answers both.
  • In the California FAIR Plan Association exam, 259 sampled files produced 38 citations for a deficient investigation against four for a deficient file.

Frequently asked questions

Decision evidence is the record that shows why an insurer was entitled to rely on a claim decision at the moment it was made. Drew Young separates it from system evidence, what the model or workflow did, and governance evidence, what controls and approvals existed. A log of the decision satisfies part of that. It does not show what the insurer examined and ruled out.

Not directly. Section 4 of the bulletin lists program-level and model-level records: the written AI Systems Program, model inventories, validation and drift documentation, and third-party contracts. The claim-level obligation comes from the unfair claims settlement practices rules in each state. A carrier can satisfy Section 4 in full and still hold no record of what was checked on a claim.

In the United States there is no single federal answer and the state picture is unsettled. An NCOIL model act would have required a qualified human professional to make any denial and barred AI from being the sole basis for a denial or adjustment. It was paused in March 2026. The consistent thread across every version: a human decision-maker, and a documented basis behind the decision.

On the Hesper platform

Hesper AI runs every stage of a claim, from first notice of loss to subrogation recovery, with investigation-grade evidence behind every decision.

The platformFNOL intakeClaims triageFraud investigationSubrogation & recovery

Keep reading

← More articles on the Hesper AI blog

See Hesper AI on your documents

Request a demo and we'll run an analysis on your real document samples.