Assessing the quality of ChatGPT responses to dementia caregivers’ questions (Preprint)

Abstract

<sec> <title>BACKGROUND</title> ChatGPT became publicly available in November 2022, and both patients and caregivers are using such artificial intelligence (AI) to answer their health questions. Studies have examined ChatGPT responses to patients’ questions about heart disease, cirrhosis, and bariatric surgery, but to our knowledge, none have looked at the quality of ChatGPT responses to topics relevant to dementia caregivers. AI such as ChatGPT holds great promise to improve dementia patients’ and caregivers’ quality of life by providing high-quality responses to their questions about typical dementia behaviors. So far, however, evidence on the quality of such ChatGPT responses is limited. </sec> <sec> <title>OBJECTIVE</title> This pilot study examines the potential of AI with CharGPT to provide high-quality information that may enhance dementia care and patient-caregiver education. </sec> <sec> <title>METHODS</title> Our interprofessional team used a formal rating scale to evaluate ChatGPT responses to real-world questions posed by dementia caregivers. For examination, we selected 60 posts by dementia caregivers from Reddit, a popular social media platform. These posts were verified by 3 interdisciplinary dementia clinicians as representing dementia caregivers’ wants and needs for information in the areas of memory loss and confusion, driving, and aggression. </sec> <sec> <title>RESULTS</title> ChatGPT quality scores ranged from 3 to 5 (highest quality); overall, 26 (43%) of the 60 responses received 5 points; 21 (35%), 4 points; and 13 (21.7%), 3 points (Table 2), suggesting high quality. </sec> <sec> <title>CONCLUSIONS</title> ChatGPT provided high-quality responses to complex questions posted by dementia caregivers, but it did have limitations. For example, ChatGPT was unable to anticipate future problems that a human professional might recognize and address in a clinical encounter. At other times, ChatGPT recommended a strategy that the caregiver had already explicitly tried. This pilot study suggests the potential of AI to provide high-quality information to enhance dementia care and patient-caregiver education in tandem with information provided by licensed health care professionals. Evaluating the quality of responses is necessary to ensure that older adults can make informed decisions; ChatGPT has the potential to transform the field of medicine by shaping how caregivers receive health information. </sec> <sec> <title>CLINICALTRIAL</title> na </sec>

Ähnliche Arbeiten