“ChatGPT can make mistakes. Check important info.” Epistemic Beliefs and Metacognitive Accuracy in Students’ Integration of ChatGPT Content into Academic Writing

Abstract

Recent studies have identified ChatGPT as an epistemic authority; however, no research has yet examined how epistemic beliefs and metacognitive accuracy impact actual work with ChatGPT-generated content, which is often prone to factual inaccuracies. Therefore, the present experimental study aimed to examine how university students integrate correct and incorrect information from expert-written and ChatGPT-generated articles when writing independently (N = 49) or with ChatGPT assistance (N = 49). Students working with ChatGPT-4o integrated more correct information from both expert-written (d = 0.64) and ChatGPT-generated articles (d = 0.95), but ChatGPT-assisted writing did not affect the amount of incorrect information sourced from the ChatGPT-generated article. Regardless of the condition, hierarchical regressions revealed that lower metacognitive bias was moderately associated with increased inclusion of correct information from the expert-written article (R2 = 12%). Conversely, a higher metacognitive bias (R2 = 10%) and epistemic beliefs (R2 = 12%) were moderately related to the inclusion of incorrect information from ChatGPT-generated articles. These findings suggest that while ChatGPT assistance enhances the integration of correct human- and AI-generated content, metacognitive skills remain essential to mitigate the risks of incorporating incorrect AI-generated information.

Ähnliche Arbeiten