A decision log tells an examiner what your system decided. It does not tell them what the claim had been checked for first. Drew Young, founder of Veriscopic, named the gap in September 2026 by splitting AI accountability into three layers: system evidence, governance evidence and decision evidence. His question: "Why was the insurer entitled to rely on that decision when it was made?"
The third layer goes missing because it costs something. A cleared check produces nothing the adjuster needs and one more line to type, so under caseload pressure it is the first thing to stop being written down. Which decisions stay with a person at all is mapped in which claims decisions stay human.
Insurance decisions have long afterlives
Answer
What is decision evidence, and how is it different from an audit trail?
Decision evidence is the record that shows why an organisation was entitled to rely on a decision at the moment it was made. Drew Young separates it from system evidence, meaning what the model or workflow did, and governance evidence, meaning what controls and approvals were in place. An audit trail is system evidence.
The framework comes from a piece Young published in Insurance Edge on 22 September 2026. System evidence and governance evidence are what most AI assurance work produces. Decision evidence is what gets tested when somebody comes back.
Most AI assurance produces the first two layers. Examiners test the third.
Drew Young's split of AI accountability: what the system did, what controls existed, and why relying on the decision was justified.
- System evidenceWhat did the model, workflow or agent do? An audit trail lives here.
- Governance evidenceWhat controls, approvals, policies or oversight were in place?
- Decision evidenceWhy was the organisation entitled to rely on this decision, in this context?
The companion piece he published on 1 September 2026 is the stronger one for claims. The evidence behind a decision may no longer exist together, "in the state in which the decision occurred", because it ends up "distributed across claims systems, underwriting platforms, emails, documents, model logs, data stores and people's memories."
For a head of claims operating under delegated authority, this is an annual event: the claims audit a capacity partner runs on a sample of files. The question in that audit is not what the decision was. It is what the file shows was looked at.
Six fields describe a decision. An investigation has more.
Answer
What should a claim decision record contain?
Five Sigma's published record carries six fields: the decision and timestamp, the inputs available, the actor with version, the delegated authority, the override path and the override history. Those are the right six for logging a decision. None of them records which checks ran on the claim and came back clean.
Five Sigma published the clearest public specification of a decision record on 17 September 2026. Its examiner framing is correct: "An examiner will ask you to reconstruct a specific decision on a specific claim on a specific date, and show who or what made it, on what basis, under whose authority, and who could have overridden it."
The list is not wrong, and a carrier that can produce it on demand is ahead of most of the market. But the object is smaller than the thing being examined. What an examiner tests is the investigation that preceded the decision.
A property claim is denied on a late-reporting ground. Three years later the policyholder's counsel asks what else the carrier considered:
- Was the prior-loss history pulled?
- Were the other addresses associated with the insured run?
- Was the weather data for the loss date checked against the reported cause?
- Was the vacancy exclusion tested and found not to apply, or was it never read?
Every one of those is an activity inside the Model 902 definition of investigation, and not one of them is a field in the log.
That gap closes only when the artefact is produced by the work rather than written up after it. Hesper leaves behind the output of every investigation phase, including the ones that came back clean. Running the phases in parallel is what makes that affordable.
The claim-file rules already ask for the activities
Answer
Do US claim-file rules require documenting the investigation or just the decision?
The investigation. NAIC Model Regulation 902 requires documentation sufficient to permit reconstruction of the insurer's activities relative to each claim. California adds a second test on top of event reconstruction: the licensee's actions must be determinable from the file. New York and Pennsylvania use the same reconstruction construction.
Four texts run a reconstruction test on the claim file, and three of them are older than any claims AI on the market. None of them uses the phrase decision evidence. All of them describe it. Pennsylvania wrote its own version of the test in December 1978.
NAIC Model Regulation 902 Section 4.B requires "detailed documentation" in each claim file "in order to permit reconstruction of the insurer's activities relative to each claim." Activities, not conclusions.
Section 3.G defines "Investigation" as "all activities of an insurer directly or indirectly related to the determination of liabilities under coverages afforded by an insurance policy or insurance contract." No distinction between activities that changed the outcome and activities that did not. A record organised around the outcome drops the larger subset.
Fifteen illustrative activities behind one liability determination. The three that moved the outcome are what a decision record keeps. The twelve that cleared are the negative space a reconstruction test asks about.
California wrote the second test explicitly. 10 CCR 2695.3(a) requires files "in such detail that pertinent events and the dates of the events can be reconstructed and the licensee's actions pertaining to the claim can be determined." A file can pass the first test and fail the second, which is the exact shape of the gap.
What a market conduct examiner actually opens
Answer
What does a market conduct examiner test on claim files?
A random sample of files, measured against standards one at a time, against a tolerance. The NAIC examination standards summary lists eleven claims standards. Two land on this subject: Standard 2, timely investigations are conducted, and Standard 5, claim files are adequately documented. They carry separate samples.
Examinations run on a tolerance rather than on perfection. A Kansas Insurance Department report of examination of the Liberty Mutual Group, covering the period to 31 December 2004, puts it plainly: "A tolerance standard of 7% is used for claim procedures and 10% is used for all other procedures." That report is twenty-two years old and tolerances vary by state, so read the figure as illustrative of the method.
The NAIC Market Regulation Handbook examination standards summary keeps the two standards apart, and that separation is what a decision log cannot survive. The log answers Standard 5 for one line item: a record exists, it is dated, it names the actor. Nothing in it describes an investigation.
A decision log answers Standard 5. Standard 2 needs the investigation record.
Two of the eleven NAIC claims standards land on this subject, and examiners test them separately, each with its own sample.
The clearest itemised public example is the California Department of Insurance targeted examination of the California FAIR Plan Association, adopted on 25 May 2022. Examiners randomly selected 259 claim files and cited 418 violations.
Thirty-eight citations for failing to "conduct and diligently pursue a thorough, fair and objective investigation". Four for failing to maintain a file detailed enough to reconstruct. The file did not fail on what was not written down. It failed on what was not checked.
Penalties attach. The Florida Office of Insurance Regulation's insurer compliance report for October 2025 lists seven property and casualty consent orders from market conduct examinations of hurricane claims handling, with fines from $50,000 to $400,000. They are not described as documentation findings, but they show these examinations carry money.
Carriers miss the investigation record because of throughput, not ignorance. A manual SIU investigation runs 14+ days per case against caseloads of 200+ cases per investigator, both Hesper internal benchmarks, so the negative findings go unwritten first. Why most flagged claims never get investigated is the same arithmetic from the capacity side.
The AI rulebook covers the model. The claim file covers the claim.
Answer
Does the NAIC AI model bulletin require insurers to document individual claim decisions?
Not directly. Section 4 of the bulletin lists what a department may request: the written AI Systems Program, model inventories, validation and drift documentation, and third-party contracts. Those are program-level and model-level records. The claim-level obligation comes from the unfair claims settlement practices rules in each state.
The NAIC adoption map, status as of 31 August 2026, records 26 jurisdictions that have adopted the AI model bulletin, plus four more with insurance-specific AI regulation or guidance of their own. A programme built against that surface is worth having. It is not the source of the claim-level obligation.
The NAIC's own issue brief of March 2026 is blunt: AI "does not alter insurers' legal obligations. Existing state insurance laws apply regardless of whether decisions are made by humans, algorithms, or third-party vendors." A carrier can satisfy Section 4 and still hold no decision evidence at claim level.
The one instrument drafted at claim level did not pass. An NCOIL model act, summarised in the minutes of the NAIC Big Data and AI Working Group meeting of 29 September 2025, would have barred AI as the sole basis for claim denials or adjustments. NCOIL paused it in March 2026. New Jersey's A5494 is the live state version, and it regulates who signs rather than what is in the file.
Negative findings are evidence, and audit practice says so
Answer
Does a claim file have to document what was ruled out?
Yes, if the file is meant to show the investigation happened. Audit practice is explicit about the equivalent duty: PCAOB Auditing Standard No. 3 requires documenting the procedures performed, identifying the items inspected, and retaining information that contradicts the final conclusion. Missing documentation casts doubt that the work was done.
Auditing wrote down what a defensible file looks like. PCAOB Auditing Standard No. 3, now AS 1215, requires that the auditor "document the procedures performed, evidence obtained, and conclusions reached" and that the documentation "clearly demonstrate that the work was in fact performed".
Paragraph 8 goes further than anything in the claims rulebook. Documentation must include "information the auditor has identified relating to significant findings or issues that is inconsistent with or contradicts the auditor's final conclusions." Contradictory evidence stays in the file by rule. Set the two records against the questions asked later.
The practical difference is production cost. A decision log is cheap because it writes down an event that already happened. An investigation record is expensive at human throughput, because every cleared check is another minute of typing that produces no benefit to the adjuster doing it.
At human throughput, the cleared checks are the first thing left unwritten
Why the industry ended up with decision logs: a capacity problem wearing a documentation costume.
- 1.A cleared check gives the adjuster nothing, and costs one more line to type
- 2.Under caseload pressure, it stops being written down
- 3.The decision survives in the file. The search that justified it does not.
A cleared check costs the same to preserve as one that hits.
Running the investigation as software changes that arithmetic. When the checks execute as agent actions, the record of each is a by-product rather than an extra task. Coverage of flagged claims moves from 25% to 100%, both Hesper internal benchmarks, and the file arrives in minutes rather than weeks.
What that file has to contain is specified in the defensibility standard for investigation AI, with the architecture underneath it in the guide to autonomous AI claims investigation.
The file you hand a regulator, a reinsurer, or a jury
Answer
Who reads a claim file after the claim closes?
Three audiences with one test between them. A market conduct examiner asks whether the investigation was conducted and documented. A capacity partner's claims audit asks what the delegated decision rested on. A court asks whether the record is trustworthy enough to admit. All three are reconstruction questions.
Admissibility is the strictest version of Young's question. Federal Rule of Evidence 803(6) admits a business record made at or near the time by someone with knowledge, kept in the course of a regularly conducted activity, where making the record "was a regular practice of that activity", unless "the method or circumstances of preparation indicate a lack of trustworthiness". Every clause is about process discipline, not about being right.
A file assembled by hand under deadline pressure and one assembled by an agent that logs every tool call face the same test, and only one carries the timestamps by construction. More in what survives a courtroom.
The capacity-partner version is the one that catches MGAs. A delegated-authority audit runs on a sample of files a year or more after the decisions in them, by someone with no access to the people who made them. A file of conclusions reads as a file of assertions.
With Hesper, agents take a claim from first notice to final recovery, and the evidence is a by-product of doing the work. The same discipline runs the whole path, mapped in the guide to automating the claims lifecycle:
- Coverage analysis produces a cited position naming the endorsements it tested and rejected.
- The investigation writes down every phase, including the ones that cleared.
- Settlement valuation carries the comparables it used and the ones it discarded.
- The recovery screen records the files where no recoverable third party was found.
This describes obligations, not compliance
Nothing here asserts that Hesper or any other system satisfies NAIC Model Regulation 902, 10 CCR 2695.3, 11 NYCRR 216.11, 31 Pa. Code 146.3 or any state's unfair claims settlement practices act. Those obligations sit on the insurer, and whether a given file meets them is a judgement for the carrier's own counsel.
Nobody decided that cleared checks should go unrecorded. Sampling-era investigation depth was a capacity constraint, not a policy choice. When the investigation runs on every file, the record of what was ruled out becomes a by-product of the work. That makes Young's third layer buildable at claim level, and the shortest route to decision evidence is producing the investigation that generates it.
Key takeaways
- Decision evidence, in Drew Young's three-layer framing, is the record of why an insurer was entitled to rely on a decision when it was made.
- The obligation is not new: NAIC Model Regulation 902 Section 4.B requires documentation that permits reconstruction of the insurer's activities, and Pennsylvania has run a version of the test since December 1978.
- Five Sigma's six decision-record fields are the right six for logging a decision, and none of them carries the checks that came back clean.
- Market conduct exams test timely investigations and file documentation as two separate standards, so a decision log answers one while an investigation record answers both.
- In the California FAIR Plan Association exam, 259 sampled files produced 38 citations for a deficient investigation against four for a deficient file.