Researching the Reliability of ChatGPT-4, 4 Plus and Google Gemini in Responding to Questions About the Management of Lower Urinary Tract Trauma: Cross-Sectional Study

Abstract

Objective: The emergence of artificial intelligence chatbots in recent years has dramatically changed the landscape of education and access to knowledge. The role of these large language models (LLM) within medicine is still relatively unclear due to their novelty. The question of whether such systems can replace the knowledge and experience of trained doctors is highly debated. Lower urinary tract trauma is a very complex emergency to manage. First responding physicians often need consultation. In this study, we aimed to investigate whether these three LLMs are reliable enough in management of lower urinary tract trauma. Material and Methods: The recommendation tables of the bladder, urethra and genital trauma management section of the European Association of Urology 2024 guidelines were analyzed. Those with strong and weak recommendation levels were translated into a questionnaire. Questions were asked in English via ChatGPT-4, ChatGPT-4 Plus and Google Gemini. Answers were evaluated by two surgeons experienced in urogenital trauma and subjected to statistical analysis by calculating the mean score. Results: In total, 27 questions were included in the study. While ChatGPT-4 and 4 Plus were more successful than Google Gemini in urethral trauma management, Google Gemini and ChatGPT-4 Plus were statistically better in approaching bladder trauma (p<0.001). In total, ChatGPT-4 and Google Gemini gave 81.4% correct and sufficient answers to the questions, while this rate was 88.8% in ChatGPT-4 Plus (p=0.618). Conclusion: Our study confirms that ChatGPT-4, 4 Plus and Google Gemini are a reliable resource in lower urinary tract trauma management. In future, it may become a platform to help healthcare professionals in determining the managing trauma.

Ähnliche Arbeiten