Evaluation of Chat Generative Pretrained Transformer (ChatGPT) Performance in Answering Kidney Transplant Related Questions

Abstract

Objective: Social media such as (Youtube, Facebook, Instagram, Twitter, etc.) and artificial intelligence (AI) are applications that have become popular in recent years, they are the first resources that patients turn to today. ChatGPT is an AI-powered language model developed by OpenAI and its success on health problems are demonstrated by many studies. In this study, we aimed to evaluate the adequacy of ChatGPT’s answers to questions about kidney transplantation. Material and Methods: Frequently asked questions about kidney transplantation by patients on health forums, websites and social media (YouTube, Instagram, Twitter) were analyzed. We also analyzed the recommendation tables of the Kidney Transplantation section of the 2024 European Association of Urology (EAU) guidelines. Those with strong recommendations were translated into a question form. ChatGPT version 4o questions were asked and the answers were evaluated by 3 urologists experienced in kidney transplantation. Results: Of the 126 questions evaluated, 65 questions were continued after the exclusion criteria. 57 (87.6%) of the answers were correct and adequate. According to EAU Guideline recommendations, 77 questions were prepared. 64 (83.1%) of the questions were answered completely correctly. There were no completely wrong answers in both frequently asked questions and questions adapted from the EAU Guidelines. Reproducibility of the questions was 100%. Conclusion: Our study confirms that ChatGPT is a reliable source for kidney transplantation. We think that it will be a platform that both patients and their relatives and healthcare professionals can frequently refer to in the future.

Ähnliche Arbeiten