AI-supported preparation may work differently when it sends students back to class with better questions. A study published in Scientific Reports on 26 October tested a programme combining AI exploration, reflection on confidence, identification of unresolved questions and an instructor’s response. Its classroom-belonging results were promising, but the experiment had only six randomised classes.[1]
The researchers assigned three undergraduate class sections to each condition at one Chinese university, with 110 students per group. They assessed students in Weeks 1, 6 and 12, beginning the intervention in Week 2. The units of randomisation were the six sections; treating the study as 220 individually randomised assignments would overstate the information in its design.[1]
The result depends on the method of inference
Week 12 belonging comparison
| Measure | Value | Context |
|---|---|---|
| Difference in change between conditions | 0.361 scale points | |
| Parametric 95% confidence interval | 0.145–0.576 | |
| Bonferroni-adjusted parametric p-value | 0.029 | |
| Exact two-sided randomisation p-value | 0.100 |
The parametric analysis suggests a difference between conditions under its modelling assumptions. The exact test asks how unusual the observed contrast is among the allocations the randomisation could have produced. With three intervention classes selected from six, there are only 20 possible balanced assignments; the two-sided test cannot resolve probabilities finely enough to go below 0.100 here.[1]
The two results therefore do not establish opposite versions of what happened. They show a favourable observed pattern and different amounts of inferential precision. The smaller parametric p-value should not conceal the limits of six clusters, while the exact p-value does not erase the measured difference.[1]
The teacher’s response is part of the intervention
The authors also report differences of 0.282 in emotional engagement, −0.497 in reactance to teacher guidance and −15.24 points in a confidence–performance discrepancy index. All primary exact tests returned p=0.100. The discrepancy index was preliminary, and exploratory pathway analyses changed with measurement timing.[1]
The package tested more than access to a chatbot. Asking students to reflect on what they know and carry unresolved questions into a responsive classroom could matter independently of the model’s answers. The study cannot assign the belonging difference to any one of these components.[1]
Sources & further reading
- Structured generative AI preparation and classroom belonging in a six-class randomized trial Scientific Reports · 26 Oct 2026 · Accessed 26 Oct 2026