Assessing ChatGPT Responses to Common Patient Questions on Knee Osteoarthritis

Abstract

Background Patient education is an important component in providing high quality healthcare, especially in the context of orthopedic surgery. In the current era of continuous technological advancements and the adoption of artificial intelligence in healthcare, the use of online chatbots in patient education is inevitable. The purpose of this paper is to evaluate ChatGPT-3.5’s effectiveness in answering common patient questions about knee osteoarthritis. Methods Ten frequently asked questions were collected from ten separate healthcare institution pages and input into ChatGPT-3.5. The questions were then analyzed for reliability and completeness using the DISCERN instrument and the Journal of the American Medical Association (JAMA) Benchmark criteria. The readability was analyzed using the Flesch Kincaid scoring system. Results Of the ten questions, the average DISCERN score was 51. Three responses were considered good, six were fair, and one was poor. The JAMA Benchmark criteria was zero for all responses. The average Flesch Kincaid grade level score was 29.33, indicating a college grade reading level. Conclusion ChatGPT-3.5 may have the potential to be an informative tool for patients with questions about knee osteoarthritis. It was able to provide fair responses, however, some inquiries required clarification and all responses lacked reliable citations. Furthermore, the responses were written at a college grade reading level, which limits its utility. Therefore, proper patient education should be conducted by orthopedic surgeons. This highlights the need for patient education resources that are both accessible and comprehensible.

Ähnliche Arbeiten