Fewer recorded legal AI errors are a signal to examine, not a measured error rate

Newly recorded US court cases involving AI-generated errors fell in the second and third quarters of 2026, Reuters reported on 23 October. That is a useful change in the visible record. Establishing whether underlying errors are falling requires knowing how detection, publication and the volume of AI-assisted work changed too.[1]

The dataset’s unit of observation is not every legal paper submitted, still less every AI-generated sentence. Charlotin catalogues judicial decisions in which courts or tribunals address hallucinated content or related AI problems, including cases where reliance on AI-generated false material was found or implied. Some allegations without confirmation are included at the curator’s discretion. The database itself says it cannot cover the larger universe of fake citations never discussed in a published ruling.[2]

A shrinking number in a growing system

At its 25 October update, Charlotin’s database listed 2,217 decisions worldwide and 1,514 tagged USA. Those are cumulative counts. The reported quarterly slowdown is a separate measure of newly identified US cases; it cannot be inferred by comparing a cumulative worldwide total with a quarterly US figure.[2][1]

Several mechanisms could move the recorded count: better checking before filing, changes in model behaviour, fewer published decisions discussing an error, or delays between a decision and its entry in the database. The tracker records the visible endpoint and cannot by itself separate those causes.[1][2]

A false citation has several lives

Suppose a model invents a precedent. The mistake can be caught by the drafting lawyer, a reviewing colleague, opposing counsel, a judge or nobody. Only a subset of these paths produces a published judicial decision discussing AI use. A database of such decisions therefore sits at the end of a long detection pipeline. A rise might mean more errors—or more vigilance. A fall might mean better practice—or fatigue. Those are not excuses to ignore the numbers; they are reasons to specify which process the numbers track.[2]

The database includes fabricated authorities, false quotations and misrepresentation. Checking that a cited case exists therefore covers only one part of the problem; the proposition attributed to it also needs support in the text.[1]

A stable, time-adjusted series would make the trend more informative. For now, the observable finding is a reported slowdown in identified court cases, with a growing cumulative database available for examining how the errors occur.[1]

Report an error or suggest a correction