Students given access to generative AI during a timed learning assignment scored better on a subsequent test taken without it. Zara Contractor and Germán Reyes report an immediate gain of 0.27 standard deviations, with a smaller benefit persisting one week later. Their experiment separates a polished assignment from the knowledge a student can still use after the assistant is removed.[1]
Zara Contractor and Germán Reyes randomly assigned undergraduate students in proctored, in-person sessions to learn an unfamiliar topic and write an analytical essay with or without access to off-the-shelf generative AI. They then tested participants without AI, both immediately and one week later. Measures included knowledge questions probing facts and concepts, as well as essays assessing higher-order abilities. That separation is the important design choice: a task completed while assistance is available is not the same as a skill retained without assistance.[1][2]
What survived when the assistant disappeared
The authors report an average immediate knowledge-test improvement of 0.27 standard deviations for participants given AI access. The gains persisted on the follow-up assessment one week later. A standard-deviation effect is a comparison to variation in the measured outcome; it is not a 27 percent improvement in marks or a guarantee that every student benefited. Without the experiment's full score distribution, converting the figure into a familiar exam percentage would be misleading.[1]
The essay results were subtler. The authors did not find a large improvement in essay quality while students had AI access. One week later, when students wrote without the tool, their essays showed gains in style and relevance. The benefits varied with how the AI had been used. Students who used it to explain and support their own understanding—the researchers' augmentation category—showed more persistent improvement. Students who used it primarily to automate the writing process lost their short-term gains when the assistance was gone.[1][2]
The difference between saving time and outsourcing thought
This is not just a debate about whether a machine writes good paragraphs. Learning has several stages: encountering information, trying to organise it, checking comprehension, rehearsing retrieval and applying ideas to a fresh problem. An assistant can remove unproductive obstacles from those stages, but it can also remove the very effort through which the learner builds a durable mental model. What the student asks for and does next may determine whether the tool functions as a tutor or a ghostwriter.[1]
AI users shifted time from drafting towards reading and searching, and reported greater enjoyment of learning. Those observations suggest ways the tool may have helped. Students were randomised to AI access, however, rather than to particular styles of use; the contrast between tutoring and delegated writing should therefore be read as an association within the experiment.[1]
A practical test for educators
A school or university that wants to assess educational value could borrow the underlying logic without copying the experiment. Distinguish assistance used during practice from independent ability after practice. Ask students to explain a concept in their own words, apply it to a new case and revisit it a week later. Review whether they interrogated an AI explanation or merely copied an output. These are suggested evaluation principles, not interventions that the paper itself tested across all possible classrooms.[1]
The experiment gives educators a more useful outcome to measure: independent understanding after assisted practice. Its encouraging result is that a tool used during study can leave a benefit after access ends.[1][2]
Sources & further reading
- Experimental Evidence on the Learning Impact of Generative AI Zara Contractor and Germán Reyes / IZA Discussion Paper 18792 · Accessed 24 Oct 2026
- Experimental Evidence on the Learning Impact of Generative AI, preprint Contractor and Reyes / arXiv · 9 Jul 2026 · Accessed 24 Oct 2026