Evaluation of the accuracy of ChatGPT-generated information in the field of general audiology

Abstract

Aim: This study evaluates the accuracy and reliability of ChatGPT’s responses to open-ended questions in otology and audiology, focusing on its potential use in training ear, nose, and throat (ENT) professionals. As artificial intelligence (AI) applications like ChatGPT become more accessible to healthcare professionals and the public, ensuring that the information provided is reliable, accurate, and reproducible is crucial, especially in the medical field. Materials and Methods: In March 2024, 60 audiology-related questions, categorized as ‘general audiology,’ ‘hearing,’ and ‘balance,’ were posed twice using ChatGPT (version 4) on the same computer to assess reproducibility. The responses were recorded as the '1st' and '2nd' answers. Three ENT specialists independently evaluated the answers to ensure accuracy, with a third reviewer specializing in audiology assessing the agreement between the responses. Answers were categorized as 1 (completely correct), 2 (partially correct), 3 (mixed accuracy), or 4 (incorrect). Analyses were conducted separately for each subgroup. Results: Statistically significant difference was found between the two responses in general audiology questions (p = 0.008) and across all responses collectively (p = 0.002), while no significant difference was observed in hearing and balance questions (p > 0.05). The second responses had higher accuracy rates, with 65%, 80%, and 70% accuracy for general audiology, hearing, and balance areas, respectively. Conclusion: ChatGPT's second responses were more accurate and reliable, making it a valuable resource for clinicians despite occasional misleading answers. With continued advancements, AI is expected to become a more reliable tool in audiology.

Ähnliche Arbeiten