A comparative evaluation of two large language models in pediatric dentistry

Abstract

In recent years, artificial intelligence and large language models (LLMs) have been developed rapidly. This study evaluated the accuracy, comprehensiveness, and performance of two LLMs in answering questions about pediatric dentistry. ChatGPT-3.5 and Gemini-1.5pro were tested on 45 questions about pediatric dentistry. These questions are classified as multiple-choice questions MCQ), True/false questions, and open-ended questions. They were directed to LLMs. Responses were recorded and scored by four pediatric dentists. Statistical analyses, including ICC analysis, were performed to determine the agreement and accuracy of the responses. The significance level was set as p < 0.050. Gemini was statistically significantly more accurate in 71.1% of the questions (p = 0.001). This difference was due to the T/F section (p = 0.001). ChatGPT gave more correct answers to the MCQ. There was no significant difference between the responses of the Gemini model to different types of questions prepared in the field of pediatric dentistry and the median scores (p = 0.062). The performance of LLMs in pediatric dentistry was satisfactory. However, further training using specific, relevant data derived from reliable sources is required. Additionally, the validity of these chatbots’ responses must be meticulously verified. LLMs provide clinical support to the pediatric dentist by providing the right information quickly, so they can help him/her to perform the most appropriate treatment in the shortest possible time and thus ensure child-patient cooperation.

Ähnliche Arbeiten