Evaluation of ChatGPT Applicability to Learning Quantum Physics

Abstract

ChatGPT is an application that uses a large language model. Its purpose is to generate answers to various questions as well as provide information, help solve problems and participate in conversations on a wide range of topics. This application is also widely used by students for the purposes of learning or cheating (e.g., writing essays or programming codes). Therefore, in this contribution, we evaluate the ability of ChatGPT to answer questions in quantum physics. That is, we develop a benchmark consisting of ten questions, whose difficulty is measured on a ten-grade scale. Then ChatGPT answers are evaluated and discussed. In this way, we can measure how well quantum-physics information is processed by this application. Our results demonstrate that ChatGPT does not notice subtle differences between physical terms, and can provide wrong answers to quantum-physics-related questions. It can also provide false mathematical formulas, claim that they are correct and confirm its answers. Note that this AI application is not sure of its answers, and in seven cases it apologizes for the first answer when a user has negated it. To sum up, AI represented by ChatGPT is only able to support students in the process of learning quantum physics at the fundamental level. Moreover, during collective exams in the future, where cheating and the use of AI by students may occur, exam questions should not be descriptive, but should be focused on solving computational problems.

Ähnliche Arbeiten