AI vs. Traditional Teaching: Evaluating PaperRater and ChatGPT in Student Engagement and Writing Achievement

Abstract

Traditional English teaching approaches, which rely on teacher-led feedback, might not meet ESL students' needs in a technology-driven the learning environment. Despite numerous studies on this topic, little is known about how AI tools improve writing proficiency and engagement in university-level ESL students.This study investigates the impact of two AI tools, namely PaperRater and ChatGPT, on students’ writing skills and engagement by thoroughly assessing students’ writing scores in two distinct writing tasks, an engagement scale, and their perceptions of the AI tools. A total of 45 students, divided into three groups (two experimental and one control) participated in the study. Students in the experimental groups received feedback from AI tools while students in the control group received traditional, teacher-led feedback. Students’ scores of pre-tests and post-tests unveiled statistically significant differences, with the control group showing a smaller decrease in the writing scores compared to experimental groups. The engagement scale and semi-structured interviews at the end of the study show that AI tools can complement traditional teaching methods, especially in increasing engagement, but that immediate performance improvements are difficult. This study has significant implications for language teachers and policymakers.