Abstract No: 284 Comparison of an Artificial Intelligence Generated Patient Education Manual with an Evidence-Informed Manual for Patients Undergoing Coronary Artery Bypass Graft Surgery

Abstract

Purpose: Artificial Intelligence (AI) is being widely used to generate patient education materials (PEM). However, this has not been studied in cardiac rehabilitation (CR). We, therefore, aimed to compare AI-generated PEM with evidence-informed material (EIM) developed for patients undergoing coronary artery bypass graft surgery. Relevance: AI could assist PEM development if the information generated is appropriate and adequately readable for patients. AI models used: Chat-GPT, Google Gemini, Medichat-Llama3. Methods: Two EIM, one each for prehabilitation and rehabilitation, were developed previously without AI assistance and evaluated for readability and appropriateness. For the AI models, two PEM each were generated using specific prompts created through an iterative process. Analysis: The AI-generated PEM underwent thematic analysis, and they were mapped to those of the EIM, European Society of Cardiology (ESC) guidelines and the Information Needs in CR (INCR) questionnaire. Corrected Covered Area (CCA) was calculated to evaluate overlap of the themes. Readability was calculated using the Flesch Kincaid Grade Level (FKGL). Results: There was a wide variation in the themes forming the different manuals, with additional themes identified in the AI-generated PEM. The AI-generated PEM did not address all the components of the ESC guidelines or the INCR questionnaire. The CCA was 0.26 and 0.22 for the preoperative and postoperative materials, respectively. The FKGL scores of the preoperative materials were 6.43, 10.55, 10.73, and 11.19 for the EIM, Chat-GPT, Gemini, and Medichat-Llama3 manuals, respectively; for postoperative materials, they were 6.47, 10.28, 9.86, and 9.44, respectively. Conclusion: AI-generated manuals, although addressing additional themes, missed out on the core components of CR. AI-generated manuals did not possess adequate readability and appropriateness for the patient population. Implications: Additional human intervention to improve readability and appropriateness of content may be required to utilize AI-generated PEM in clinical practice.

Ähnliche Arbeiten