iWrite study finds no positive link from reported reflection to criticism

Students' positive views of an automated essay scorer do not necessarily extend to greater scrutiny of its feedback. A study of 442 English-as-a-foreign-language students at Chinese private universities found several positive relationships among perceptions of iWrite, but did not support the proposed positive links from innovativeness and reflective thinking to reported criticism. Published on 29 October, it examines attitudes to the system, rather than demonstrating that students became better writers or better judges of automated feedback.[1]

The researchers asked students about how interactive iWrite felt, whether it performed as expected, confirmation of expectations, individual openness to innovation, reflective thinking and their criticism of the scoring system. They examined the relationships through partial least squares structural equation modelling, or PLS-SEM. This statistical method can fit a proposed network of associations among measured attitudes, but a cross-sectional questionnaire is not equivalent to assigning comparable students randomly to different types of feedback and measuring what they learn.[1]

The pattern was not a simple chain from useful feedback to critical independence. Perceived interactivity was positively related to perceived performance and confirmation. Perceived performance was also positively related to confirmation. Students who reported stronger confirmation tended to report greater innovativeness, reflective thinking and criticism. Innovativeness in turn was positively associated with reflective thinking. Yet the proposed positive direct links from individual innovativeness and reflective thinking to criticism were not supported. In the fitted model, being open to the tool and saying that it prompted reflection did not automatically mean students challenged it more.[1]

Feedback satisfaction is not a literacy test

A learner can agree that a scoring service responds quickly or offers actionable suggestions while having little basis for judging whether a particular correction improves meaning, style or argument. Conversely, a student who questions a score may be perceptive, confused or responding to a genuinely poor explanation. The survey's construct called criticism must be read as a self-reported attitude within the authors' questionnaire, not an independently scored demonstration of the ability to identify faulty automated feedback. The reported questionnaire analysis does not establish whether participants could successfully reject erroneous suggestions in an independent test.[1]

The findings illuminate a familiar problem in educational technology: the convenience of a feedback loop can be mistaken for evidence of deeper learning. Automated essay scoring can return comments immediately and repeatedly, an advantage when instructors face large classes. But a numerical grade or a fluent explanation is only one input into writing development. A student also needs to know why a recommendation is appropriate for the essay's audience and argument, which suggestions are optional, and when the model's apparent certainty exceeds the available evidence.[1]

An instructor seeking to assess an AI scoring tool could provide students with several comments of mixed quality, including deliberately misleading feedback, and measure whether students can justify accepting or rejecting them. That would directly evaluate critical use. A longitudinal comparison could also measure the quality of revised essays without AI assistance after several weeks of feedback, rather than focusing solely on whether students liked the original interaction. Such designs would test a learning claim that the iWrite survey, by itself, cannot resolve.[1]

The article's setting limits how broadly its coefficients should be generalized. Respondents came from Chinese private university EFL contexts, and language proficiency, the assessment culture, prior familiarity with writing technologies and the particular scoring interface may shape students' answers. The researchers' structural paths are correlations in a specified model. They do not establish that increasing interactivity will cause criticism, or that a different generative writing assistant would produce the same pattern. Even a statistically significant path can be educationally modest if it does not translate into stronger writing or judgment.[1]

There is also a useful distinction between productive scepticism and universal distrust. Students should not have to dismiss every automated suggestion to prove independence; they should be able to explain which feedback advances their communicative aim and which does not. That requires examples, feedback provenance, teacher involvement and opportunities to revise decisions. A system can help deliver these experiences, but an agreeable user interface cannot stand in for them.[1]

For universities, the next evaluation could ask students to explain which automated comments improve an essay and which do not, then assess writing completed without the tool. That would connect perceptions of an interactive service to the capabilities education is meant to develop.[1]

Report an error or suggest a correction