Assessing Quality, Readability, and Transparency of Online and Artificial Intelligence-Generated Information on Type 2 Diabetes Mellitus: A Cross-Sectional Study (Preprint)

Abstract

<sec> <title>BACKGROUND</title> Type 2 Diabetes Mellitus (T2DM) affects approximately 590 million people worldwide, and its management relies heavily on patient education. With the emergence of online health information and Artificial Intelligence Large Language Models (AI LLMs), patients are increasingly sourcing medical information independently. </sec> <sec> <title>OBJECTIVE</title> This study compares the quality, readability and reliability between traditional online resources and AI-generated information related to T2DM. </sec> <sec> <title>METHODS</title> Four predefined search terms were entered into three major search engines (Google, Yahoo, and Bing), and the top 20 search results were retrieved. Patient information AI-generated leaflets (AIGLs) were produced using a standardised prompt across four AI LLMs (ChatGPT, Gemini, DeepSeek, and Grok). Information quality was assessed using the DISCERN score and was calculated by the author and ChatGPT. The JAMA benchmark was used to measure reliability and transparency. The Flesh Reading Ease Score (FRES) and Flesh-Kincaid Grade Level (FKGL) were used to determine the readability and comprehension. </sec> <sec> <title>RESULTS</title> Eighty websites and four AIGLs were analysed, with author-rated mean DISCERN scores of 43.6 (±10.9) and 43.8 (±2.986), mean JAMA Benchmark scores of 2.74 (±0.965) and 0, mean FRES of 50.6 (±14.4) and 48.9 (±9.16), and mean FKGL scores of 8.66 (±2.23) and 8.3 (±1.92), respectively. The ChatGPT-rated mean DISCERN scores for websites and AIGLs were 58.5 (±11.5) and 61.0 (±2.94), respectively. </sec> <sec> <title>CONCLUSIONS</title> Given the high prevalence of T2DM, both traditional online and AI-generated T2DM resources demonstrate suboptimal quality, accessibility, and transparency. Increasing patient reliance on digital health information calls for improved readability standards and stronger safeguards for AI-generated content. The landscape of medical consultations is evolving, with patients increasingly presenting with preconceived notions based on online health information; hence, healthcare professionals should adapt to this shift. </sec>

Ähnliche Arbeiten