AI does not only invent facts. It can change what people remember seeing

A person watches a car approach an intersection. A day later, they read a summary that names the wrong traffic sign. When asked what they saw, many remember the sign from the summary rather than the one in the video. The failure is no longer confined to the AI output: the generated text has become part of the person’s memory of the event.[1][2]

In the 2026 experiment, 83.6% of participants who received an accurate summary correctly recalled the sign. Accuracy fell to 44.8% after a misleading summary—a decline of 38.8 percentage points. Telling participants that the summary came from AI rather than a professional human transcriber did not significantly protect them. Familiarity with AI and reported trust in it also failed to eliminate the effect.[1]

The summary arrived after the experience—and altered the recollection

The study first tested commercial multimodal models on two short animated accident videos. Across twenty generated summaries, every summary contained at least one error. Omissions were dominant: the systems left out 51.6% of the details the researchers had classified as central, and nineteen of twenty summaries omitted the collision between the car and pedestrian. The memory experiment then isolated one misleading detail so the researchers could test its effect rather than the general quality of the summaries.[1]

Why the effect survives an AI warning label

Human memory is reconstructive. Later information can be woven into the account of an earlier event, especially when it is plausible and fills a gap. A source label changes a person’s stated reason for trusting a document; it does not necessarily stop the detail from becoming familiar. When the memory test arrives later, the participant may retrieve the summary’s version without a clear tag showing where it came from.

This creates a flaw in the usual “human in the loop” defence. If the reviewer saw the original event and then reads the AI summary, the reviewer is not standing outside the system. The summary can alter the very memory being used to check it. A person may confidently “correct” the record toward the machine’s error rather than away from it.[1][4]

The safest workflow separates evidence from narrative

The studies do not establish that every AI summary implants memories or that every corrected image is harmful. They use controlled materials, particular misleading details and online participant samples. The 2026 summary experiment used two simple animated videos and deliberately constructed conditions; it does not measure the everyday error rate of police, medical or workplace systems. The value of the evidence lies in the demonstrated mechanism, not a universal percentage.[1]

AI-generated text is often treated as a disposable convenience layer over an authoritative record. These experiments show that it can become part of the cognitive record too. Once the summary has been read, returning to the source may not restore the reviewer to the mental state they had before. The operational safeguard is therefore temporal: preserve and inspect first, summarize second, and keep the boundary between what happened and what the machine said happened visible throughout.

Report an error or suggest a correction