Assessing the Performance of ChatGPT in Addressing Questions Regarding Urethral Strictures a Cross Sectional Study

Abstract

Objective: ChatGPT is an artificial intelligence program with natural language processing. In this study, we aimed to evaluate the accuracy and reproducibility of ChatGPT's answers to frequently asked questions (FAQs) about urethral strictures. Material and Methods: FAQs about urethral strictures on the websites of urological associations and medical establishments were analyzed. Also, strong recommendation-level information was gathered from the urethral strictures section of the European Association of Urology (EAU) 2022 Guidelines. All questions were asked in order in ChatGPT August 3rd version. All answers were evaluated separately by two specialist urologists and scored between 1-4. Where 1: completely correct, 2: correct but insufficient, 3: combination of accurate and misleading information, and 4: completely incorrect. This study was conducted in accordance with the principles of the Declaration of Helsinki. Results: Of the FAQs, 89.5% were answered completely correctly. No question was answered completely incorrectly. All questions about general information, prevention and complications were graded as 1. Of the 45 questions prepared according to the EAU guideline recommendations, 34 (75.6%) were evaluated as Grade 1, and 5 (11.2%) and 3 (6.6%) as Grade 3 and 3 (%6.6) as Grade 4. All questions related to general information, prevention and complications received the same answer the second time they were asked. Conclusion: Our findings demonstrated that ChatGPT accurately and satisfactorily answered majority of the questions about urethral strictures. Although it has limitations, ChatGPT is predicted to have an important position in the health sector as it is a constantly evolving platform.

Ähnliche Arbeiten