In Reference to <i>The Comparative Diagnostic Capability of Large Language Models in Otolaryngology</i>

Abstract

We read the article of Warrier et al. entitled “The Comparative Diagnostic Capability of Large Language Models in Otolaryngology.1” The study addresses a significant necessity in our domain as we contend with the swift incorporation of artificial intelligence (AI) into clinical practice. The authors' rigorous methodology in examining ChatGPT-3.5, Google Bard, and Bing-GPT4 using 100 clinical vignettes establishes a solid framework for evaluating these tools. Their discovery that ChatGPT-3.5 surpassed its peers with a 95.7% accuracy rate (excluding instances for further testing) is both remarkable and stimulating. We congratulate the authors for their timely and interesting study comparing the diagnostic capacities of large language models (LLMs) in otolaryngology.1 This work well illustrates the potential of AI in otolaryngology. Some recent works demonstrated several potential applications of ChatGPT-4 in the ENT field, in particular finding LLM's reliability in analyzing laryngeal pictures.2, 3 These papers jointly highlight the increasing significance of AI in clinical decision assistance and teaching. Nevertheless, it is essential to regard these findings with tempered hope. Kleebayoon and Wiwanitkit underscored the necessity for careful incorporation of ChatGPT in clinical otolaryngology,4 a viewpoint reiterated by Tessler et al.5 in their assessment of ChatGPT's compliance with clinical practice guidelines. The performance diversity among various LLMs, as emphasized by Warrier et al., accentuates the necessity for careful assessment and oversight in their clinical utilization. A limitation of the study is its emphasis on diagnosis accuracy, neglecting the intricacies of clinical reasoning and the possibility of AI augmenting, rather than supplanting, human competence. Subsequent study may gain from integrating measures that evaluate the quality and pertinence of AI-generated explanations, as examined by Zalzal et al. in their assessment of ChatGPT's capacity to respond to patient inquiries.6 Furthermore, the swiftly advancing characteristics of AI technology present a problem for comparative analyses. We would underscore the essential requirement for standardized evaluation instruments in measuring AI efficacy in medical applications. The Artificial Intelligence Performance Instrument (AIPI)7 represents an important advancement in this area. Integrating validated instruments in future studies would improve the comparability and reliability of findings across various AI platforms and medical specializations. In the final analysis, Warrier et al. have offered significant insights on the present capabilities and constraints of LLMs in otolaryngology diagnoses. As we advance the integration of AI in our domain, research such as this will be crucial in directing responsible application and pinpointing areas for further enhancement. Sincerely, Antonino

Ähnliche Arbeiten