The Impact of Large Language Models (Llms) Like Chatgpt on Student Creativity

Abstract

The active integration of generative artificial intelligence systems into the field of education and the mixed results of their use in the socio-humanitarian sphere raise questions about the quality of their impact on creativity as a universal competency of personal productivity. This study focuses on exploring the influence of large language models (LLMs), such as ChatGPT, on student creativity. Objective. To examine the impact of LLMs (ChatGPT) on student creativity in solving academic tasks from the perspective of objective assessment and subjective self-evaluation of its manifestation. Materials and Methods. The sample consisted of two groups of first-year philosophy students - an experimental group (EG) and a control group (CG), each comprising of 30 students. Both groups completed four creative tasks as part of their academic coursework; the EG used LLMs (ChatGPT) for idea generation and information retrieval, while the CG did not use LLMs. The results were assessed using five creativity criteria based on J.P. Guilford and E.P. Torrance's models (uniqueness, non-obviousness, novelty of approach, transformation, and quantity of ideas), a "Self-Assessment of Creativity" questionnaire with open-ended questions, and the MAI-32 "Metacognitive Awareness Inventory" test. Qualitative analysis was conducted using content analysis; statistical processing was performed using variance analysis; and effect sizes (Cohen's d and Hedges' g) were calculated to assess the significance of differences. Results. The hypothesis that LLMs positively influence student creativity was not supported. Objective assessment showed higher creativity results in the CG, which did not use LLMs. Moreover, in two tasks, a significant negative effect was observed, possibly due to fixation on the technique of working with the tool. Paradoxically, subjective self-assessment of creativity was statistically significantly higher in the EG, which used LLMs, resembling a "placebo effect," as the process of working with LLMs was perceived as more creative and productive at the selfperception level, despite objectively lower results.Conclusion. The active imagination of students without the use of LLMs yields objectively higher results across all creativity criteria and induces critical self-reflection. To enhance the effectiveness of LLMs, it is advisable to introduce them as a "timely prompt" only after intellectual and emotional frustration arises, which will stimulate the awakening of creative intuition.

Ähnliche Arbeiten