Comparative Efficacy of General Versus Specialized AI in Emergency Decision Support for Advanced Dementia: A Caregiver-Centric Simulation Study

Abstract

Informal caregivers of individuals with advanced dementia often face acute behavioral and medical crises requiring urgent decision-making. While general-purpose large language models (LLMs) like ChatGPT offer on-demand support, their efficacy in high-risk scenarios remains uncertain. This study compares general and specialized AI models in emergency dementia caregiving support. A four-phase simulation study was conducted, beginning with real-world emergency scenario collection from 58 caregivers (yielding 52 unique prompts). Four AI models—ChatGPT-3.5, GPT-4, retrieval-augmented GPT-4o, and a domain-constrained "PDC30 Chatbot"—were evaluated using blinded expert and caregiver reviews. Responses were scored on accuracy, guideline concordance, safety, empathy, and completeness, with errors classified by severity. Specialized AI models outperformed general-purpose LLMs, with the domain-constrained PDC30 Chatbot achieving the highest scores in guideline adherence (4.2/5) and safety (5.0/5) while producing zero critical errors. General models (ChatGPT-3.5/GPT-4) exhibited more critical errors (4–7) and lower completeness scores (2.7–3.2). Retrieval-augmented GPT-4o also showed significant improvements over general AI, particularly in clinical concordance (3.9/5). Clinically adapted AI systems provide safer, more reliable emergency support for dementia caregivers than general-purpose LLMs. Integration of evidence-based guidelines and domain constraints is critical to minimizing risks in AI-assisted caregiving.

Ähnliche Arbeiten