Giving students access to an AI tutor did not produce better grades in a randomized University of Maryland trial. An October 2026 working paper reports lower final grades when comparing sections of the same course, alongside substantially lower participation in the learning-management system. The result concerns one course-integrated deployment, not the educational value of every AI tutor.[1]
The study involved 2,379 undergraduates and 30 instructors during autumn 2025. Its same-course analysis contained 1,353 students. Keeping those samples separate matters: the adjusted grade reduction in the full sample was not statistically significant at the conventional 5% level.[2]
Two estimates, with different levels of certainty
| Analysis | Adjusted grade difference | Statistical evidence |
|---|---|---|
| All 2,379 students | −2.88 points on a 0–100 scale | p < 0.10; not p < 0.05 |
| 1,353 students in matched course sections | −4.12 points on the same scale | p < 0.05 |
Only about 15% of students offered the assistant used it at least once. Both groups could access other AI tools. The assistant also generally operated in direct-instruction mode rather than a mode designed to guide students through questions. Those details restrict the claim: this was not a comparison between intensive guided tutoring and a world without AI.[2]
Access is the intervention
Low uptake does not automatically invalidate a randomized access study. It changes the question the study answers. A university deciding whether to introduce a service needs to know what happens when it is offered under ordinary conditions, including whether students ignore it. That is different from asking how much a carefully supervised session can teach a student who completes it.
Nor can the observed grade difference simply be divided by the percentage who used the tool to calculate harm per user. Introducing an assistant might change expectations or behaviour beyond recorded sessions. Separating those pathways would require stronger assumptions or a different experimental design.
Engagement logs are a clue, not a diagnosis
The paper’s headline same-course estimates express the grade effect as a decline of 0.37 standard deviations and the participation effect as 0.90 standard deviations. The authors suggest that the assistant’s introduction may have displaced existing learning activities. The public abstract presents that as a possible interpretation, not an observed account of each student’s learning process.[1]
A lower number of clicks is not inherently worse learning. A student might need fewer pages because an explanation was effective, or might avoid useful practice because an answer was immediately available. The accompanying grade results make the engagement decline important, but they do not identify which explanation caused it.
For instructors, the useful design question is therefore specific: what work should the learner still do after asking for help? An assistant that supplies a solution, one that asks for an attempted step and one that generates a new practice problem are different educational interventions even when they use the same underlying model.
The Maryland findings justify evaluating the instructional arrangement as carefully as the chatbot. Future comparisons could hold course content constant while varying how assistance is delivered and checking later independent performance. A helpful answer in the moment and durable learning are different outcomes; this working paper is a reminder to measure the latter rather than assume it.
Sources & further reading
- The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement: A Randomized University Trial Annenberg Institute, EdWorkingPapers · Accessed 27 Oct 2026
- EdWorkingPaper 26-1598: full paper and Table 3 Jing Liu and colleagues, University of Maryland · Accessed 27 Oct 2026