Written by Human or ChatGPT - Authorship Forensics in the Era of Generative AI

Abstract

In this evolving landscape of text generation, distinguishing between human-written and ChatGPT-generated content has become increasingly important. This paper presents a novel approach to authorship attribution, leveraging both Statistical Natural Language Processing (SNLP) and Convolutional Neural Networks (CNN) techniques to differentiate between documents written by humans and ChatGPTs. The research uses 212 abstracts of academic papers written and published by the research group as the human-written set and asks both ChatGPT 3.5 and 4 to generate corresponding abstracts based on paper titles as AI-written set. Models are trained on a laptop to classify human and AI-written abstract texts in 2-class (i.e., human and ChatGPT) and 3-class (i.e., human, ChatGPT 3.5, and ChatGPT 4) based on their part-of-speech tag frequency distribution patterns. The 2-class model is well-trained in less than ONE minute (i.e., 56.82 seconds) and the 3 -class model is welltrained in 7 minutes and 26.076 seconds. The results demonstrate a significant ability of the models to distinguish between human and AI-written text, with precision 0.9682 (F0.5 score 0.95) for the 2-class (human and ChatGPT) testing subset and precision 0.9806 (F0.5 score 0.96) in the 3-class (human, ChatGPT 3.5, and ChatGPT 4) testing subset. The proposed 3 -stage Authorship Forensics approach has been implemented as an open access web application to allow teachers and users to either train their own models or use the existing trained model to get some advice on how the model considers a piece of given text written by human or AI.

Ähnliche Arbeiten