Evaluation of the ChatGPT for Fixed Orthodontic Treatment

Abstract

The aim of the current study is to evaluate the quality, reliability and readability of data provided by ChatGPT-4, ChatGPT-3.5 in the field of fixed orthodontic treatment. Guidelines on fixed orthodontic treatment were reviewed and 20 questions were listed by two researchers for patients to ask chatbots. Answers were obtained from ChatGPT-3.5 and ChatGPT-4 by two different researchers and 3 different scoring criteria, Reliability Scoring System (adapted from DISCERN), Global Quality Scale (GQS) and Simple Measure of Gobbledygook (SMOG) were used to evaluate the chatbot's answers to the 20 questions. A statistically excellent level of consistency was found between the 2 researchers. Statistically significant differences were found between the groups in all 3 categories (DISCERN, GQS, SMOG) revealed statistically significant differences among chatbots (p = 0.001, p=0.008, p<0.001, respectively). For ChatGPT-4, 3 categories were calculated as 2.25±0.44, 4.30±0.571, 19.3±1.24; for ChatGPT- 3.5, 1.70±0.470, 3.70±0.733, 17.7±1.23 respectively. For GQS values, ChatGPT-4 is considered as good quality and ChatGPT-3.5 as medium. According to DISCERN, ChatGPT-4 is more reliable than ChatGPT-3.5. According to the SMOG index, both chatbots require a graduate level education for readability. In terms of reliability and quality, both AI-based chatbots showed low-medium reliability, good quality and difficult readability. Although the medical information provided by chatbots in the field of fixed orthodontic treatment is of good quality, it is recommended that individuals consult their healthcare professionals.

Ähnliche Arbeiten