Assessing written texts: Can university students identify whether texts are written by humans or AI?

Abstract

In an era where AI is becoming more sophisticated and accessible, can students reliably distinguish between human-generated and AI-generated texts? This question underscores the need to incorporate generative AI platforms into learning environments to aid in increasing digital literacy (Creely et al., 2025). For example, AI chatbots are used to provide corrective feedback on foreign language writing tasks (Yang & Chen, 2025; Ziqi et al., 2024; Zou et al., 2024). The goal of this study is to determine whether university students can identify AI-generated texts and whether previous experience with the software, familiarity with AI, and location play a role. This was tested across 78 university students at two universities (NSBM Green University in Sri Lanka, Rovira and Virgili University in Spain). The students completed a pre-questionnaire, which was designed to collect background variables, including their use and knowledge of AI systems. They were then asked to read a series of texts and indicate whether they believed the text was written by a human, written and then translated with machine translation, or written by a chatbot (ChatGPT). The results from the students in Spain and Sri Lanka show that there are no significant differences in the responses. The participants could not distinguish between the human and AI-generated texts (choosing correctly only 30-40% of the time across all sections), which would suggest that the writing capabilities of AI systems are advanced enough that, even to those who believe they are familiar with the technology, they could not distinguish whether something is human-created or artificially created. These findings are relevant to educators interested in incorporating training or practice with Generative AI into their teaching practices.

Ähnliche Arbeiten