Is it possible for ChatGPT to mimic human annotator?

Abstract

The development of large language models, such as ChatGPT (GPT-3.5) and GPT-4, has revolutionized natural language processing (NLP) and opened up new possibilities in various fields. These models demonstrate remarkable capabilities in generating coherent and contextually relevant text, making them suitable for a wide range of applications. This work focuses on automatic text annotation in subjective problems and person-alization using ChatGPT. The primary objective is to investigate the ChatGPT generative capabilities and evaluate its performance in classification and regression NLP tasks. Furthermore, the work also contributes a novel methodology for evaluating personalized ChatGPT and adapting it to address specific problem domains. The results obtained from multiple experimental setups showcase the potential of the method to automatically exploit ChatGPT to generate text annotations. However, the conclusions drawn from the research highlight the need for a further detailed and more extensive analysis across multiple problem domains and diverse datasets.

Ähnliche Arbeiten