Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators

Abstract

With the increasing development of artificial intelligence (AI), educators are exploring various AI-driven tools to enhance language teaching and learning processes. This study investigates the reliability of ChatGPT as an AI-powered assessment tool in the context of foreign language (FL) education. Using a mixed-methods approach, the research examines ChatGPT’s evaluative abilities compared to human evaluators, focusing on its assessment of students’ writing skills. The study employed a total of two research samples: one comprising eight students and another consisting of eight teachers of English language at the university where the research took place. The findings reveal that while ChatGPT demonstrates potential in providing feedback, its limitations, particularly in accurate assessment of grammatical nuances, emphasize the need for supplementation with human evaluation. The method section provides detailed information on the data collection process, including the procedures followed for evaluating student summaries and conducting interviews with teachers. The study highlights the importance of acknowledging AI’s limitations and adopting a complementary approach to its integration in language education.

Ähnliche Arbeiten