Performance of ChatGPT in dentistry

Abstract

Artificial Intelligence-based language model, ChatGPT, has gained significant traction due to its communicative interface and the relevance of the responses generated. This tool exhibits tremendous potential to be utilized in dentistry for dental education, and possibly as a clinical decision support system. Hence, it is imperative to evaluate the accuracy of the model in relation to the responses generated for dental-related queries and outline the limitations for its use in clinical practice. This study aims to evaluate the performance of ChatGPT-generated responses to questions from multiple dental specialties for their accuracy, completeness, and relevance. Methods: This multi-centric study involved 27 subject experts from nine dental specialties of various institutions and 2 heads of institutions. A total of 243 questions were formulated and the answers generated by ChatGPT (version: 3.5) were rated in terms of accuracy (6-point Likert), completeness (4-point Likert), and relevance (5-point Likert). Results: The mean accuracy of the ChatGPT-generated answers was 4.61 (SD 1.575), with a median of 5.33. For completeness, the mean score was 2.01 (SD 0.793), and the median was 2.33. Regarding relevance, a mean of 3.13 (SD 1.590) and a median of 3.67 were obtained. The highest ratings were observed for answers related to Oral Medicine and Radiology, as well as for open-ended questions, and questions labelled as easy in terms of difficulty. Conclusion: The promising results observed in the study promote the application of ChatGPT for retrieving dental information. However, it is crucial to exercise caution and seek advice from a qualified healthcare for dental health-related queries. Further large-scale testing of the model is necessary before incorporating it into dental clinical practice.

Ähnliche Arbeiten