---
title: "Legacy rules-based systems vs. autonomous AI: why SIUs accept only 36% of automated fraud referrals"
description: "SIUs accept 36% of referrals from automated fraud detection tools, against 70% from adjusters (CAIF and Aon, 2024). Why rules-based detection flags so many legitimate claims, what the false positive numbers really measure, and how autonomous AI investigation changes the economics of SIU work."
date: "2026-04-17"
lastModified: "2026-11-16"
author: "Pankaj Dhariwal"
tags: ["Guides"]
canonical: "https://gethesperai.com/blog/legacy-rules-vs-autonomous-ai-fraud-detection/"
---

# Legacy rules-based systems vs. autonomous AI: why SIUs accept only 36% of automated fraud referrals

> **TL;DR** No published study measures a single false positive rate for insurance fraud detection. The best public proxy: SIUs accept 36% of referrals from automated fraud tools, against 70% from adjusters (CAIF and Aon, 2024). Rules are tuned for recall, and investigation capacity caps the rest. Autonomous AI investigation works every flagged claim into a cited report in minutes, not weeks.
>
> - Automated tools send 45% of SIU referrals; only 36% of them are accepted.
> - Capacity is the cap: roughly 25% of flagged claims get a full investigation.
> - Manual SIU takes 14+ days a case; one investigator carries 200+ open cases.

- **36%** - Automated fraud referrals accepted by SIUs (Vs. 70% of adjuster referrals (CAIF and Aon, 2024))
- **45%** - SIU referrals generated by automated tools (Up 6 points since 2022, while SIU staff grew 0.5% in 2024)
- **25%** - Flagged claims that receive full investigation (Hesper internal benchmark; the other 75% close without one)
- **Minutes** - Autonomous AI investigation time (Vs. 14+ days for a manual SIU investigation)

Rules-based fraud detection vs AI comes down to what each hands the SIU. Every carrier running a fraud detection stack - FRISS, Shift Technology, Verisk, or an internal rules engine - knows the trade-off. The system surfaces suspicious claims. Alerts pile up. Investigators work a small slice and close the rest without resolution.

The reason is structural. Rules-based detection was engineered to be sensitive, not precise. Autonomous AI investigation sits downstream of it: instead of handing humans alerts to evaluate, it works each flagged claim into cited findings. For the economics of uninvestigated claims, see [how uninvestigated claims drain profitability](/blog/insurance-claims-leakage-reduce-losses/).

## The false positive problem

False positives are not a measurement error. They are a direct consequence of how rules-based detection works. A scoring model checks a claim against a library of red flags - late reporting, high loss amount, recent policy inception, prior claims history. Each match adds to a risk score, and claims above a threshold trigger an alert.

But 'suspicious' and 'fraudulent' are not the same thing. A legitimate claimant who reports a theft 10 days after it occurred because they were travelling looks, to a scoring model, identical to one who waited in order to fabricate the loss.

The cleanest public measure of the gap comes from the [Coalition Against Insurance Fraud](https://insurancefraud.org) and Aon. In their 2024 benchmark of 35 property and casualty carriers, SIUs accepted 36% of referrals sent by automated fraud detection tools and 70% of referrals sent by adjusters.

Put another way, of every 100 automated referrals, an SIU accepted 36 and turned away 64 (derived). Acceptance only earns a referral an investigation; it is not confirmed fraud.

> **Why precision is capped**
>
> A rules-based system sees ~30 signals per claim: claim data, policy data, prior claims, basic external data. That is not enough to distinguish 'unusual' from 'fraudulent'. The distinguishing evidence typically lives in documents, statements, public records and OSINT sources, which require investigation, not scoring.

The consequence is capacity erosion. Every declined referral still cost someone a review, and every accepted one competes for the same investigators. Most carriers work only the highest-severity alerts, which is why roughly 75% of flagged claims are never fully investigated (Hesper internal benchmark).

**Figure 1 (Referral acceptance) - SIUs accept 36 of every 100 automated fraud referrals.** Adjuster referrals clear SIU intake at nearly twice that ratio, and an accepted referral has only earned an investigation, not a fraud finding.

100 times 36% equals 36

- Automated fraud referrals: 100
- Referral acceptance ratio: 36%
- Accepted for investigation: 36 (The other 64 are turned away at intake)

**Adjusters:** Referrals from adjusters were accepted at 70% in the same study, down from 76% in 2022.

**Flagged: Then capacity rations the accepted queue.** Only the highest-severity alerts get worked in full. The rest close with an abbreviated review or none.

- roughly 75% of flagged claims are never fully investigated (Hesper internal benchmark)

Source: Coalition Against Insurance Fraud and Aon, 2024 Insurer SIU Benchmarking Study, 35 P&C carriers. The 64 turned away is derived from the 36% referral acceptance ratio. Roughly 75% of flagged claims never fully investigated is a Hesper internal benchmark.

## What the published data says about false positives

**Is there a published false positive rate for insurance fraud detection?**

No industry study publishes one. The closest public measure is the referral acceptance ratio: in the Coalition Against Insurance Fraud and Aon's 2024 benchmark of 35 carriers, SIUs accepted 36% of automated fraud referrals and 70% of adjuster referrals. Acceptance is not confirmed fraud, so treat any single false positive figure with caution.

Round numbers such as 60% circulate widely, including in an earlier version of this guide. We could not trace any of them to a published insurance study, so we use the one open dataset on referral quality across carriers: the [sixth edition of the Coalition's SIU benchmark](https://insurancefraud.org/wp-content/uploads/2024-Insurer-SIU-Benchmarking-Study.pdf), conducted for 2024 by Aon.

Automated tools now send 45% of all SIU referrals, against 49% from adjusters. Yet they account for just 29% of accepted referrals, while adjusters account for 60%. Acceptance for automated tools improved from 32% to 36%; for adjusters it slipped from 76% to 70%. As [Insurance Business noted on November 9, 2026](https://www.insurancebusinessmag.com/us/news/breaking-news/automated-tools-now-make-45-of-siu-referrals--investigators-havent-kept-pace-596528.aspx), that gap suggests automated alerts are less likely than other referrals to be accepted for investigation.

| Share of SIU referrals sent vs accepted, by source (2024) | Value | Share |
| --- | --- | --- |
| Automated fraud detection tools: share of all referrals | 45% | 45% |
| Automated fraud detection tools: share of accepted referrals | 29% | 29% |
| Adjusters: share of all referrals | 49% | 49% |
| Adjusters: share of accepted referrals | 60% | 60% |

Read the numbers for what they are. A declined referral is not proof of a legitimate claim, and an accepted referral is not proof of fraud. What the data shows is direction: machines send nearly half the referrals, those referrals clear intake at about half the adjuster rate, and SIU headcount rose just 0.5% in 2024.

## Hit rate, acceptance, confirmation: which false positive rate?

**Why do fraud detection vendors' accuracy figures not compare?**

Because each divides a different pair of numbers. A vendor hit rate counts flags a reviewer finds worth investigating. A referral acceptance ratio counts what SIU intake takes on. A confirmation rate counts investigations that substantiate fraud. A strict false positive rate counts legitimate claims that got flagged.

Shift Technology says its Force solution delivers a 75% hit rate, and a [profile published by Guidewire](https://www.guidewire.com/resources/blog/industry-trends/driving-efficiencies-through-better-fraud-detection-shift-technology) defines that hit rate as the percentage of claims the solution deems potentially fraudulent that the insurer ultimately deems suspicious and worthy of investigation upon human review.

That vendor-reported figure sits at the same stage as SIU acceptance, not confirmation. Neither it nor the 36% benchmark says how many flags ended in proven fraud.

The vocabulary is muddled too. As [Google's machine learning course defines them](https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall), the false positive rate is the share of actual negatives wrongly classified as positive; precision is the share of positives that are correct. "Most fraud flags are false positives" describes low precision, not a high false positive rate.

| Metric | What it divides | Where it is measured | What it cannot tell you |
| --- | --- | --- | --- |
| Vendor hit rate | Flags a human reviewer deems worth investigating / all flags | Vendor and carrier, each on its own definition | Whether any fraud was confirmed |
| Referral acceptance ratio | Referrals the SIU accepts / referrals sent | SIU intake; benchmarked by CAIF and Aon | What the investigation found |
| Confirmation rate | Investigations that substantiate fraud / investigations closed | SIU case management | Fraud in flags that were never worked |
| Precision | Flags that are fraud / all flags | Only where every flag gets an answer | Fraud the screen never flagged |
| False positive rate | Legitimate claims flagged / all legitimate claims | Model validation on labelled history | How crowded the SIU queue will be |

## How rules-based detection works

Modern fraud detection tools combine rules, statistical models and network analysis. The core output - a risk score and an alert - is consistent across vendors. Claims above a configured threshold route to the SIU queue, where an investigator evaluates the alert and decides whether to investigate.

This pipeline does what it was designed to do: surface suspicious activity for human evaluation. It is not designed to confirm fraud, gather evidence or resolve cases. That work happens downstream, in the SIU.

Legacy is not necessarily a pejorative. Rules-based detection is fast, interpretable, auditable and regulator-friendly, and [FRISS](/compare/hesper-vs-friss/), [Shift Technology](/compare/hesper-vs-shift-technology/) and Verisk are category standards for precisely these reasons. The question is not whether detection is useful. The question is whether detection is sufficient.

## The base-rate arithmetic behind rules-engine false positives

**Why does a fairly accurate fraud screen still produce mostly false alarms?**

Because fraudulent claims are a small minority of the book. When honest claims outnumber fraudulent ones many times over, even a small error rate on the honest side produces more wrong flags than right ones. Tightening the threshold trims the false alarms but lets more fraud through, so precision and recall trade off.

The numbers below are hypothetical, chosen for round arithmetic, not an industry fraud rate. Take a book of 10,000 claims in which 300 are fraudulent. Suppose a rules screen catches 90% of the fraud, so 270 fraudulent claims get flagged, and wrongly flags one in ten of the 9,700 honest claims, which is 970 more.

The SIU queue now holds 1,240 alerts, and only 270 of them are fraud. That is about 22% precision from a screen that is right on 90% of each kind of claim.

Now tighten the screen so it flags only one in twenty honest claims, and assume it catches 75% of the fraud as a result. The queue shrinks to 710 alerts and precision rises to about 32%. The price is that 75 fraudulent claims slip through instead of 30.

Rules or machine learning, a recall-tuned screen on a mostly honest book produces mostly false alarms. Better models move the curve; they do not abolish it. The threshold is a choice about which error to pay for.

## How autonomous AI investigation works

Autonomous AI investigation agents operate downstream of detection. The input is a flagged claim from any detection source - rules engine, adjuster referral, external tip. The output is an investigation-ready report with confirmed findings, evidence and a recommendation.

**Figure 2 (Downstream of detection) - The agent investigates the flag. The investigator decides.** Any detection source can feed it; the output is an investigation-ready report, and every determination stays human.

Flagged claim: Suspected auto injury, from any source (Rules_engine_alert.json, Medical_records.pdf, Recorded_statement.mp3)

**Investigation agent** (15+ phases in parallel, every finding cited)

1. Document forensics on records, estimates and photos
2. Medical records checked against the injury mechanism
3. NICB, ISO ClaimSearch and state DMV cross-referencing
4. Statements and EUOs cross-referenced for consistency
5. Timeline rebuilt from every source
6. Network check for rings and provider collusion

200+ signals per case, not ~30

Hand-off: Cited report

**Investigator decides** (Human accountability for fraud findings)

- Final fraud determination
- Claim denial
- SAR filing
- Recovery actions

Agents recommend. Investigators approve.

**Not a swap:** FRISS, Shift Technology and Verisk keep detecting; the agent starts where their alert ends.

Source: Illustrative decision-rights map for a suspected auto injury claim. Phases as listed in this article; human sign-off on fraud determinations per state DOI requirements and the NAIC model SIU regulation.

(Illustrative example)

A properly designed autonomous agent runs 15+ investigation phases in parallel, from document forensics and medical record analysis to statement analysis and timeline reconstruction.

That is 200+ data points per case rather than the ~30 signals a rules engine evaluates. A detection system tells you a claim is suspicious. An investigation agent tells you why - with citations, evidence and a structured timeline.

> The false positive rate conversation stops being interesting once you automate investigation. If every flagged claim gets fully investigated in minutes, it no longer matters whether detection was 20% precise or 60% precise. You find the real cases, and the false ones are resolved as part of the workflow.
>
> - Hesper AI product research, Q1 2026

## Head-to-head comparison

Detection and investigation are not competing categories - detection sits upstream of investigation - but they are frequently evaluated together. Traditional fraud analytics vendors provide detection and scoring but do not automate the full investigation workflow. The table below summarises the difference.

| Dimension | Detection: rules and scoring (FRISS, Shift, Verisk) | Autonomous AI investigation (Hesper) |
| --- | --- | --- |
| Primary output | Risk score and alert | Investigation-ready report |
| Workflow stage | Pre-investigation (detection) | Investigation (resolution) |
| Signal density per case | ~30 signals | 200+ signals |
| How quality is measured | Referral acceptance: 36% for automated tools (CAIF and Aon, 2024) | Outcome per claim, every finding cited |
| Time per case | Seconds (scoring) | Minutes (full investigation) |
| Human time required | 4-8 hours per investigation | human review |
| Investigation coverage | Capacity-limited to ~25% of flagged | 100% of flagged claims |
| Cost per investigation | High (investigator time + vendors) | A fraction of the manual cost |
| Replaces human decision | No | No |
| Regulator-ready output | Requires investigator write-up | Structured report with citations |

The comparison is not a claim that investigation makes detection more precise. It is about who decides which flags get an answer. Today that is the capacity ceiling: an accepted referral still waits its turn for an investigator.

When investigation runs on every flagged claim, the declined and never-worked flags get an evidence-backed answer too. Fraud sitting in the unworked majority of flags gets examined instead of closed out.

## What changes in the SIU workflow

Autonomous investigation does not replace SIU teams. It changes what investigators do. The shift is from execution to judgement.

The manual SIU workflow is execution-heavy. Investigators spend an estimated 88% of their time on evidence gathering, documentation, administrative logging and vendor coordination. A manual investigation takes 40-80 hours of investigator time plus vendor fees for surveillance, medical peer review, and database pulls. On a Hesper internal benchmark, an investigator carrying 200+ active cases completes roughly 10 investigations per month at 14+ days each.

**Figure 3 (Before: investigator as executor) - Investigators spend 88% of their time on execution, not judgement.** Where a manual investigator's hours go today, and what one manual investigation costs in time.

**Manual investigator time**

- Evidence gathering, documentation, logging, vendor coordination: **88%** - What is left goes to analysis and decision-making

**One manual investigation**

- **40-80** - hours of investigator time, plus vendor fees
- **14+ days** - per manual SIU investigation

**Takeaway:** The shift is from execution to judgement: the agent gathers the evidence, the investigator reviews it and decides.

Sources: this article's estimate of manual SIU time allocation (estimated, not a time-and-motion study); 40-80 investigator hours and 14+ days per manual investigation as stated in this article.

With autonomous investigation agents, the AI handles planning, evidence gathering, analysis and report generation. The investigator reviews findings, makes the final determination, and coordinates with claims, legal and law enforcement where required. Throughput is bounded by review time, not by the days a manual investigation spends gathering evidence.

| Activity | Manual SIU | With AI investigation |
| --- | --- | --- |
| Evidence gathering | 5-15 days per case | Minutes (parallel, automated) |
| Document forensics | Manual review, selective | Pixel-level on every document |
| Database queries | Manual, sequential | Automated, parallel |
| Statement cross-referencing | Investigator reads and compares | Automated with cited inconsistencies |
| Report writing | 4-8 hours per case | Auto-generated; human review |
| Investigator cases/month | ~10 (Hesper internal benchmark) | Bounded by review time |
| Investigation coverage | 25% of flagged claims | 100% of flagged claims |

> **What stays human**
>
> The final decision on every claim remains with a human investigator. Autonomous agents surface findings, cite evidence and recommend outcomes. They do not deny claims, file SARs or trigger recovery actions autonomously. Regulators and claims law require human accountability for fraud determinations; autonomous investigation preserves it.

## How to cut false positives without missing fraud

**How can insurers reduce false positives in fraud detection?**

Route alerts by weight instead of sending every hit to the SIU, feed every investigation outcome back into the rules and models, and govern any model that replaces rules under your state's AI guidance. The step most carriers skip is getting an answer on every flag, because unworked alerts teach a screen nothing.

Start with routing. In [Insurance Business's November 2026 interview](https://www.insurancebusinessmag.com/us/news/breaking-news/automated-tools-now-make-45-of-siu-referrals--investigators-havent-kept-pace-596528.aspx) with Coalition Against Insurance Fraud leaders, Brent Walker, the Coalition's director of government relations and a former senior SIU leader, recommends tiers. Low-weight hits stay with a claims professional or underwriter, mid-tier hits go to desk or field investigators, and the most complex go to major case units.

Next, close the loop. Walker adds that investigators' outcomes have to feed back into the model so it stays accurate and avoids bias. That only works if outcomes exist: a flag closed without review produces no label. An agent that works every flagged claim to a cited finding turns the whole queue into labelled outcomes.

Finally, govern the replacement. The [NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers](https://content.naic.org/sites/default/files/cmte-h-big-data-artificial-intelligence-wg-ai-model-bulletin.pdf.pdf), adopted December 4, 2023, names fraud detection as an AI use and expects a written AIS Program for AI systems that make or support regulated insurance decisions, with due diligence and audit rights over third-party systems.

As of November 3, 2026, 29 jurisdictions had adopted the bulletin and four more, California, Colorado, New York and Texas, had their own insurance-specific guidance, per the [NAIC adoption map](https://content.naic.org/sites/default/files/legal-adoption-map-ai-model-bulletin.pdf). Whatever does the flagging, keep the fraud determination with a person.

## Evaluation criteria for buyers

Whether AI-driven fraud infrastructure replaces legacy detection or, more commonly, complements it, these criteria matter most. They map to the questions decision-makers ask during RFP and procurement.

1. Scope: ask what the system produces. If a score, you are buying detection. If a report, you are buying investigation.
2. Signal density: rules-based detection evaluates ~30 signals. Autonomous investigation should evaluate 200+.
3. Evidence and auditability: every finding should be backed by cited evidence. Ask to see a sample investigation report with full citations.
4. Integration: the agent should sit downstream of whatever detection stack is in place, without requiring replacement.
5. Data retention: expect a contractually defined window, deletion on written instruction, and no training on carrier data.
6. Regulatory posture: confirm the vendor's workflow preserves human sign-off on fraud outcomes.
7. Deployment time: legacy fraud platforms typically take 6-18 months to deploy. A well-designed investigation agent should be in production on real claims within 30-90 days.
8. Measurement: investigation measures outcome - confirmed fraud per flagged case, investigation coverage rate and time-to-close.

## Key takeaways

- No published study gives a single false positive rate for insurance fraud detection. The best public proxy: SIUs accept 36% of automated fraud referrals versus 70% from adjusters (CAIF and Aon, 2024).
- Vendor hit rates, referral acceptance and confirmation rates measure different stages. Ask which one you are being quoted before comparing numbers.
- False positives are only a crisis because manual investigation capacity is limited. Roughly 25% of flagged claims get a full investigation (Hesper internal benchmark).
- Autonomous AI investigation agents sit downstream of detection and investigate flagged claims end-to-end: 200+ signals per case, turnaround in minutes, investigation-ready report. The investigator keeps the decision.
- Detection and investigation are complementary, not competing. Most carriers will keep their detection stack and add autonomous investigation downstream.

Related reading: [why 75% of flagged claims are never fully investigated](/blog/why-flagged-insurance-claims-never-investigated/), [how insurance companies investigate fraud: inside the SIU process](/blog/how-insurance-companies-investigate-fraud/), and [Hesper AI vs. manual investigation](/compare/hesper-vs-manual-investigation/).

## Frequently asked questions

### What percentage of insurance fraud alerts are false positives?

No industry study publishes a single figure, and round numbers such as 60% do not trace to a published insurance source. The closest public measure is the Coalition Against Insurance Fraud and Aon's 2024 Insurer SIU Benchmarking Study of 35 property and casualty carriers: SIUs accepted 36% of referrals from automated fraud detection tools and 70% from adjusters. Acceptance means a referral earned an investigation, not that fraud was confirmed.

### What is the difference between fraud detection and fraud investigation?

Fraud detection surfaces suspicious claims for human evaluation. It produces a risk score and an alert. Fraud investigation confirms whether fraud actually occurred and produces evidence that supports a claim decision. Detection takes seconds. Traditional manual investigation takes 14+ days. Autonomous AI investigation compresses it to minutes while preserving human decision-making.

### Does autonomous AI investigation replace FRISS, Shift Technology, or Verisk?

No. FRISS, Shift Technology, and Verisk are detection platforms - they identify suspicious claims and route them to SIU teams. Autonomous AI investigation agents like Hesper AI sit downstream of detection and investigate the flagged claims. Most carriers deploy both. The two categories are complementary: with both, 100% of flagged claims can be investigated rather than ~25%.

### Are autonomous AI investigation agents regulatory-compliant?

Properly designed autonomous investigation agents are compliant with state DOI fraud regulations and the NAIC model SIU regulation. The key requirement is that human investigators retain decision-making authority on fraud determinations - agents produce findings and recommendations, not autonomous denials. Suspicious activity reports (SARs) remain a human-signed regulatory filing.

### What should I look for when choosing an AI claims investigation partner for an SIU team?

Focus on scope - detection only, or detection plus investigation with investigation-ready output; evidence and auditability - every finding should have citations that hold up to regulator and litigation scrutiny; integration - the system should sit alongside your existing detection stack rather than requiring replacement; and measurement - confirmed fraud per flagged case, investigation coverage rate, time-to-close, and leakage reduction.
