Human vs. AI: Assessing the Quality of Weight Loss Dietary Information Published on the Web

Abstract

Information availability through the web has been both a challenge and an asset for healthcare support, as evidence-based information coexists with unsupported claims. With the emergence of artificial intelligence (AI), this situation may be enhanced or improved. The aim of the present study was to compare the quality assessment of online dietary weight loss information conducted by an AI assistant (ChatGPT 4.5) to that of health professionals. Thus, 177 webpages publishing dietary advice on weight loss were retrieved from the web and assessed by ChatGPT-4.5 and by dietitians through (1) a validated instrument (DISCERN) and (2) a self-made scale based on official guidelines for weight management. Also, webpages were assessed by a ChatGPT custom scoring system. Analysis revealed no significant differences in quantitative quality scores between human raters, ChatGPT-4.5, and the AI-derived system (p = 0.528). On the contrary, statistically significant differences were found between the three content accuracy scores (p < 0.001), with scores assigned by ChatGPT-4.5 being higher than those assigned by humans (all p < 0.001). Our findings suggest that ChatGPT-4.5 could complement human experts in evaluating online weight loss information, when using a validated instrument like DISCERN. However, more relevant research is needed before forming any suggestions.

Ähnliche Arbeiten