Can the ChatGPT and other Large Language Models with internet-connected database solve the questions and concerns of patient with prostate cancer?

Abstract

Abstract Large language models (LLMs), such as ChatGPT, have shown impressive natural language processing capabilities in various fields, including medicine. However, the answers provided by these models may sometimes be incorrect, and they may not have access to the latest data. In this study, we aimed to evaluate the performance of five state-of-the-art LLMs in providing correct and comprehensive information on common questions raised by prostate cancer patients. We also examined whether LLMs with internet-connected databases could provide more up-to-date information than ChatGPT. We designed a set of 22 questions covering various aspects of prostate cancer and evaluated the accuracy, comprehensiveness, patient readability, and inclusion of humanistic care in the answers provided by each model. Our findings suggest that although the performance of different LLMs varied, these LLMs could provide accurate basic knowledge and have the ability to analyze specific situations to a certain extent. We also found that the overall performance of the LLM model with internet-connected dataset was not superior to ChatGPT, and the paid version of ChatGPT did not show superiority over the free version. Our study highlights the potential of LLMs in bridging the gap between patients and healthcare providers. Current LLMs have the potential to be applied for patient education and consultation, providing patient-friendly information. Shared decision-making with the doctors and patients could be achieved easier. We believed that with the rapid development of AI technology, LLMs have unlimited potential.

Ähnliche Arbeiten