Accuracy of AI Chatbots in Asthma Care: A Comparative Study of ChatGPT, Gemini, and DeepSeek

Abstract

AI-driven chatbots are increasingly used in healthcare to provide critical medical information, yet their responses can appear accurate while harboring errors that may impact patient care, particularly for asthma management. <bold>Aims</bold> to evaluate the accuracy and comprehensiveness of responses generated by ChatGPT-4-turbo, Gemini 2.0 (Google AI), and DeepSeek-V3, as of January 27, 2025, in addressing medical queries related to asthma management. <bold>Materials and Methods:</bold> Ten open-ended asthma-related questions were formulated based on the Global Initiative for Asthma FAQs and answered using ChatGPT-4-turbo, Gemini 2.0 (Google AI), and DeepSeek-V3 on January 27, 2025. Four pulmonologists assessed response accuracy and completeness using predefined Likert scales. Answers were rated on a six-point accuracy scale (1–6) and a three-point completeness scale (1–3). Unanswered questions received a score of 0. <bold>Results.</bold> <fig><object-id>erj;66/suppl_69/PA6250/F1</object-id><object-id>F1</object-id><object-id>F1</object-id><graphic></graphic></fig> <bold>Conclusion:</bold> The study highlights the need for continuous improvement of AI-driven language models, especially in critical healthcare areas like asthma management. While these models provide valuable information, inconsistencies in accuracy and completeness emphasize the importance of expert oversight when relying on AI-generated health data.

Ähnliche Arbeiten