Comparative Analysis of Responses From Five Popular Artificial Intelligence Chatbots to the Most Commonly Searched Keywords About Apheresis

Abstract

People are increasingly using artificial intelligence (AI)-based chatbots to provide health-related information. However, concerns remain regarding the quality, accuracy, and readability of the information they produce. This study aimed to evaluate and compare the responses of five widely used AI chatbots to the most frequently searched keywords about apheresis. On May 1, 2025, the 25 most searched apheresis-related keywords were identified using Google Trends. Two keywords were excluded due to irrelevance. The remaining 23 queries were submitted to five chatbots: GPT-4o, Gemini 2.5, Grok 3, DeepSeek v3, and Copilot. Responses were assessed using the EQIP tool for content quality, the DISCERN questionnaire for information reliability, and the Flesch-Kincaid grade level (FKGL) and reading ease (FKRE) metrics for readability. Statistical analysis was performed using the Kruskal-Wallis test and Bonferroni correction. Significant differences were found among chatbots in EQIP, DISCERN, FKGL, and FKRE scores (p = 0.001). DeepSeek v3 demonstrated the highest quality and accuracy (EQIP: 95.7%, DISCERN: 71.8), while GPT-4o had the best readability (FKRE: 43.1, FKGL: 9.1). Copilot showed the poorest readability. Overall, chatbot responses were generally written at a college reading level. AI chatbots vary substantially in the quality and comprehensibility of their health information about apheresis. While newer models like DeepSeek offer improved informational accuracy, readability remains a concern across all platforms. Future chatbot development should prioritize plain-language communication to enhance accessibility and health literacy for diverse patient populations.

Ähnliche Arbeiten