#1901 Evaluating ChatGPT for dietary support: photo-based ingredient recognition for dialysis patients

Abstract

Abstract Background and Aims An adequate diet is critical for dialysis patients to manage nutrients like protein, potassium, phosphorus, and sodium. To support this, an efficient method to identify the types of food consumed in patients' daily lives is essential for accurately translating food intake into nutrient levels. The aim of this study is to evaluate the use of ChatGPT for image recognition and its ability to correctly identify ingredients from food images. This approach could provide a valuable tool for assisting in dietary assessment and management for dialysis patients, as current methods rely heavily on self-reporting and manual review, which are labor-intensive and difficult to scale for real-time and large-scale applications. Method This study integrated GPT-4o, an optimized version of OpenAI's model with enhanced capabilities for analyzing and responding to a combination of data types, into the process of ingredient recognition from photos of prepared dishes. The model was applied to a diverse range of cuisines, including Mexican, Western, and Chinese. By incorporating various culinary styles, the study aimed to evaluate the model’s performance across dishes with different ingredient combinations and presentation styles. Photos from nine distinct dishes were provided to GPT-4o, with each image analyzed individually. The model was tasked with identifying all ingredients in each dish. To ensure accuracy and relevance, only ingredients confirmed to be present in the images were included in the evaluation (e.g., broccoli would be included whereas salt will be excluded). A matching score was computed for each dish to quantify the model’s performance. This score was defined as the ratio of correctly identified ingredients to the total number of ingredients present in photo. Results Among the nine dishes analyzed, each contained at least five visible ingredients, with GPT-4o successfully recognizing all ingredients in six of them (Table 1). Three instances of false recognition are illustrated in Fig. 1. The photos demonstrating suboptimal performance in ingredient recognition are presented to highlight specific challenges encountered during the analysis. Overall, the average matching score across all dishes was 94.97%. Conclusion This study investigated the ability of GPT-4o to recognize ingredients from photos of prepared dishes representing a variety of cuisines such as Chinese, Mexican, and Western. The results demonstrate that this approach can accurately identify ingredients, achieving an average matching score of 94.97%. However, challenges remain with certain scenarios, such as overlapping ingredients, misclassification of visually similar items, and the recognition of boiled or blended components. These findings highlight GPT-4o's potential as a tool for ingredient recognition, thus improving patients’ ability to better understand nutritional content of their meals.

Ähnliche Arbeiten