Nearly every student tried the AI tutor, but many left it unused when they made a maths mistake. In a two-year study of Khanmigo across 18 Tennessee middle schools, the median student consulted the tutor in only 17% of practice sessions containing an error. The programme produced modest achievement gains while exposing a gap between access to help and engagement with it.[1]
Economists Philip Oreopoulos and Nina Low studied the programme during existing daily remedial mathematics sessions. Students in assigned groups used Khan Academy with Khanmigo configured to coach rather than supply answers. Their NBER working paper reports both achievement outcomes and the interactions intended to support them.[1]
Achievement improved, with several measures of the effect
Achievement effects reported by the authors
| Measure | Estimate | Population or interpretation |
|---|---|---|
| Assignment effect per term | About 1.3 national percentile ranks | Average effect of assignment to the programme |
| Assignment effect per school year | About 0.06–0.08 standard deviations | Annual scale of achievement gains |
| Implied full-year active-participation effect | About 0.14 standard deviations | Different participation-based estimate |
The authors describe the gains as resembling those from Khan Academy practice without AI. That comparison does not establish that Khanmigo and a non-AI version were randomly assigned head to head in this experiment. The evidence available here supports an effect of the programme and raises a separate question about the conversational tutor’s contribution.[1]
A tutor one click away can still be ignored
Ninety-six per cent of students tried Khanmigo at least once. The median student messaged it on about a third of practice days, with use falling to 17% of sessions that contained a mistake. When students did engage, exchanges often consisted of brief answers or suggested prompts rather than sustained discussion. First use and meaningful help-seeking therefore describe different levels of uptake.[1]
The difference matters because a good human tutor does more than stand available in the corridor. Teachers may notice a persistent misconception, redirect attention or decide that a student should explain their reasoning aloud. A chatbot that waits to be summoned can miss the same learning opportunity if a child does not recognise when help would be useful. The researchers identify engagement as the likely binding constraint, a diagnosis grounded in behaviour rather than in model benchmarking.[1]
A programme delivered within ordinary school routines
The intervention combined practice software, a coaching-style chatbot and existing remedial maths sessions. It operated within classrooms rather than replacing teachers. A more proactive tutor might engage students differently after an error, but that is a new intervention to test, not an effect demonstrated by this study.[1]
The cluster-randomized design and two-year duration make the findings more informative than a short survey of enthusiastic users. The working paper remains evidence from a particular school setting. Its results do not establish how the same tutor would perform in other subjects, age groups or teaching arrangements.[1]
The next experiment should test the moment of difficulty
There are testable improvements suggested by these data. A student might receive a contextual invitation to ask why an answer was wrong, or the system might adapt its dialogue to invite an explanation rather than a single click. Those designs would also need to avoid excessive interruptions or doing the work for the learner. Teachers should remain able to see how students are thinking, not merely how long the software was open.[1]
Sources & further reading
- One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment National Bureau of Economic Research · Accessed 26 Oct 2026