Assessment of ChatGPT vs. Bard vs. guidelines in the artificial intelligence (AI) preclinical management of otorhinolaryngological (ENT) emergencies

Abstract

Background: Artificial intelligence (AI) based platforms are gaining popularity with increasing application across all areas of life. There is expanding research into AI’s ability to act as a medical resource for clinicians, patients, and their families, with emerging potential application in answering medical questions and vignettes. This study seeks to assess ChatGPT’s performance in otorhinolaryngological (ENT) emergency case vignettes, with direct comparison to Google’s Bard AI and readily available online patient resources. Methods: Sixteen short fictional case vignettes describing potential ENT emergencies were entered into ChatGPT and Google’s Bard in triplicate followed by an online search for patient resources. Results were presented in terms of diagnostic accuracy, recommendation to seek medical review, appropriateness of triage categorisation, and appropriate of preclinical measures. Results: Both AI systems diagnosed 100% of conditions with appropriate triage in 76.7%. All instances suggested seeking face-to-face medical review. Appropriate preclinical measures were outlined in 84.4% of instances, with scoring detailing no statistical difference between groups (P=0.5634) for grouped data with the Kruskal-Wallis test. Conclusions: The implications of this study suggest that AI systems cannot yet replace medical review, but may help augment patient understanding. We continue to recommend consultation with medical practitioners in all cases of ENT emergencies. Future directions can lead to further assessment of AI accuracy with other aspects of health advice and diagnosis. Providers should display transparency and disclose when AI is used to generate advice, and the issue of who would hold legal responsibility for adverse outcomes remains unexplored. AI should not be used as a stand-alone primary resource for ENT emergencies and should not replace seeking medical review. With further development, there is no doubt AI will play a greater role in pre-clinical management of ENT emergencies.

Ähnliche Arbeiten