Predicting Student Academic Performance Based On A Neural Network Classification Model Of Discussion Activity With ChatGPT

Abstract

This study explores the feasibility of using conversation complexity between students and ChatGPT as a predictor of academic performance. Drawing on interaction logs from 76 undergraduate students in a Data Science course, five conversation attributes were extracted: average word count, prompt word count, number of prompts, discussion continuity score, and cognitive level based on Bloom’s taxonomy. A neural network classifier was applied to predict academic performance labels, which were derived using unsupervised K-Means clustering on Mid-Semester Exam (UTS) scores. The model achieved an accuracy of $\mathbf{6 5. 7 9 \%}$ and an $\mathbf{F 1}$-score of $\mathbf{6 6. 7 5 \%}$ for the “poor” class and $64.86 \%$ for the “good” class. While results indicate moderate predictive potential, limitations related to data validity, feature objectivity, and exam fairness warrant caution. This study contributes to the emerging field of AImediated learning analytics by offering a pilot exploration of ChatGPT conversation data as a digital learning trace.

Ähnliche Arbeiten