After AI-assisted fraction practice, adults solved 71% unaided versus 77% for controls

A peer-reviewed paper presented at the Conference on Language Modeling in October 2026 examined what happens when AI assistance is removed after a short practice session. Across three randomised experiments with 1,222 recruited participants, access improved immediate performance but was followed by weaker results on unassisted tasks. In the fraction replication, analysed participants with AI access solved 71% of final questions compared with 77% for controls.[1][2]

In the first experiment, participants worked through fraction problems with or without a GPT-5 assistant in a sidebar. The assistant was preloaded with each problem and its solution, so participants could obtain a correct answer with a minimal request. This tests readily available answers rather than an assistant restricted to hints. After practice, the assistance disappeared unexpectedly and the final three questions were completed unaided. Among the 305 analysed participants, those assigned AI solved 57% of the last questions, versus 74% for controls; the respective skip rates were 20% and 11%. The research team acknowledged a design problem, however: excluding participants on the basis of practice-stage performance could differentially remove less-skilled control participants while retaining AI-assisted users whose answers appeared correct.[1][2]

The replication repaired a methodological weakness

The second trial enrolled 667 people, added an independent fraction pretest and gave controls a sidebar containing previously seen solutions so that the interface change itself was more comparable. After pre-specified exclusions, 585 remained in the analysis. AI users correctly solved an average of 71% of final questions, against 77% for controls, a six-percentage-point difference the authors reported as statistically significant (P=0.019). This is a more defensible estimate of the short-term independent-performance gap than the larger 17-point difference in the initial experiment.[1][2]

The second trial also recorded a higher mean skip rate among AI users, 10% rather than 7%, but that difference was not statistically significant (P=0.239). It is therefore inaccurate to present every experimental result as independently proving that AI made people give up more. The authors found that the weaker later performance and reduced persistence were concentrated among participants who chose to obtain direct AI answers. Such behaviour is observational within the randomised AI arm: people who ask for answers may differ from people who ask for hints, so that subgroup association is not a separately randomised estimate of the effect of requesting a direct answer.[1][2]

The third experiment used a different kind of work: reading comprehension. It likewise found weaker unassisted performance in the AI group after supported work. Across experiments, the design supports a short-term causal comparison of access to the particular assistants, interfaces and tasks used. It does not directly measure whether a worker who has used AI for years has permanently lost skill, or whether carefully designed tutoring prompts would have the same effects.[1][2]

What an educator or manager can reasonably change

In-session task completion and retained skill are different outcomes. A training exercise can measure both by checking how well participants solve comparable problems after assistance is removed. That design would reveal whether an apparent productivity gain also supports independent performance.[1][2]

An instructor might ask learners to attempt a problem independently before opening the assistant, require them to explain intermediate steps, or include a short no-AI assessment after a supported practice session. Those are plausible safeguards, not treatments whose effectiveness this paper independently proved. A workplace can make the same distinction between using AI to finish a job and verifying an employee's retained ability to diagnose, review or continue it.[1][2]

The study's strongest contribution is its counterfactual: the same task looks different depending on whether the evaluation stops while the assistant is present or begins when it is removed. The researchers demonstrate a measurable short-run performance trade-off in controlled settings, while leaving substantial questions about longer-term learning, different user populations and better-designed scaffolding open for further trials.[1][2]

Report an error or suggest a correction