Dies ist eine Übersichtsseite mit Metadaten zu dieser wissenschaftlichen Arbeit. Der vollständige Artikel ist beim Verlag verfügbar.
Reliability of ChatGPT-5.0 as an Automated Essay Scoring Tool: What Matters?
0
Zitationen
1
Autoren
2025
Jahr
Abstract
The aims of the present study were twofold: Exploring the variabilities of ChatGPT-5.0’s capabilities of rubric-based essay scoring across three prompting designs and two essay feeding methods; and testing the reliability of ChatGPT-generated scores against human ratings. Drawing upon three reliability measurements, including: Spearman’s correlations, Intraclass Correlations (Koraishi, 2024; Bui & Barrot, 2025) and quadratic weighted kappa (QWK) ( Mizumoto & Eguchi, 2023; Poole & Coss, 2024), the findings revealed that although the reliability coefficients ranged from moderate to substantial, the essay scoring abilities of ChatGPT-5.0 depended greatly upon users’ expertise to engineer prompts and their choices of essay feeding. This study highlights the importance of continued effort in the validation of this technology as an automated essay scoring tool and emphasizes the irreplaceability of human professional judgment in this field.
Ähnliche Arbeiten
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
2019 · 8.260 Zit.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
2019 · 8.116 Zit.
High-performance medicine: the convergence of human and artificial intelligence
2018 · 7.493 Zit.
Proceedings of the 19th International Joint Conference on Artificial Intelligence
2005 · 5.776 Zit.
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
2018 · 5.438 Zit.