The Use of Artificial Intelligence for Medication Support in Dentistry: A Reliability Assessment of Chatbots

Abstract

Objective: This study aimed to evaluate the performance of artificial intelligence (AI)-based large language models (LLMs) in providing medical treatment recommendations for clinical scenarios in dental practice, focusing on rational prescribing and drug interaction management. Methods: Forty standardized clinical case questions were developed by experienced oral and maxillofacial surgeons and submitted to ChatGPT-3.5 (OpenAI), ChatGPT-4o (OpenAI), and Gemini 2.5 Flash (Google DeepMind). Responses were generated in Turkish using standardized prompts and were independently assessed by two blinded evaluators based on three criteria: modified Global Quality Score (GQS), accuracy, and completeness. Data were statistically analyzed using non-parametric methods. Results: Gemini 2.5 Flash achieved the highest performance across all criteria, with 45% of responses rated as being of very high quality (GQS score 5), 67.5% rated as highly accurate (scores of 5-6), and 42.5% rated as complete. ChatGPT-4o outperformed ChatGPT-3.5 in all parameters but did not differ significantly from Gemini. Statistically significant differences were observed between ChatGPT-3.5 and Gemini for GQS (p

Ähnliche Arbeiten