Detectability Analysis of AI-generated Research Articles

Abstract

The rapid surge of AI-generated content, driven by advanced large language models (LLMs) such as ChatGPT, threatens to undermine academic integrity and redefine the boundaries of plagiarism in scholarly work. This article evaluated the efficacy of five publicly available AI content detection tools in differentiating human-generated content from AI-generated ones. During the assessment, an AI-authored article was analysed using Sapling AI, Turnitin, GPTZero, Copyleaks, and Quillbot tools to assess their effectiveness in detecting AI-generated content. Following the detection reports, the paper was manually revised and resubmitted to the same tools to assess their ability to detect altered AI-authored content. This approach highlighted the susceptibility of these tools to modified texts and emphasised the difficulties in reliably detecting AI-created content. The analysis was conducted using the tools in their default state as black-box systems, without a thorough analysis of their underlying mechanisms or algorithms. Furthermore, the evaluation is confined to a single article on a specific topic, and as such, the performance of the tools assessed cannot be regarded as representative. To gain a comprehensive understanding of the effectiveness of these tools, further research and assessment involving articles from diverse disciplines and a variety of writing styles is essential.

Ähnliche Arbeiten