ChatGPT and Unified State Exam in Computer Science

Abstract

In this paper, it is presented the obtained and studied statistics of solving tasks of demo versions of the Unified State Exam (USE) in Computer Science 2011-2023 using the GPT-3.5 language model and ChatGPT. The obtained results of the Unified State Examination in Computer Science are presented, their analysis and the results of solving individual tasks are shown, examples of successful solutions of the Unified State Exam tasks in Computer Science, limitations when working with ChatGPT are described. Based on the results of solving exam tasks, ChatGPT scored 47-57 test scores in 2011-2014 before the cancellation of the test part, and also slightly overcame the threshold score in 2015-2017, 2019, 2020, did not score the points necessary for passing the USE in Computer Science in 2018, 2021-2023. Based on the obtained research data, a gradual complication of the USE exam model in Computer Science is shown, in which the test part of the exam in 2015 is abandoned and the computer format of the exam is introduced in 2021. Using the example of the USE in Computer Science, it is shown that ChatGPT, GPT-3.5 and similar language models can serve tool for expert assessment of the complexity of examination tasks and examination model.

Ähnliche Arbeiten