An AI-Powered RAG Conversational Agent for Enhancing Patient Recall of Surgical Consultations: A Proof-of-Concept Study

Abstract

BACKGROUND: Surgical consultations contain dense information that patients struggle to recall, and existing tools do not provide access to individualized clinical dialogue. Retrieval-Augmented Generation (RAG) can ground language-model responses in source transcripts, enabling patients to revisit the specific advice given during their consultation. OBJECTIVES: This study evaluates the technical feasibility and performance of a RAG conversational agent built on synthetic plastic surgery encounters. METHODS: Twenty simulated patient cases were generated across ten plastic surgery procedures, each containing four sequential visits. Transcripts were converted to audio and transcribed using Whisper to simulate real-world ambient capture. The transcriptions were ingested into a Vertex AI RAG system powered by Gemini 2.0 Pro. Two hundred patient-style questions were developed across domains including procedural details, risks, recovery, medications, and clinical parameters. Responses were evaluated against gold-standard transcript answers for accuracy, precision, recall, and error type. Readability was assessed using standard metrics. RESULTS: The system achieved 99.0% accuracy (198/200), with perfect precision and 0.99 recall. One question demonstrated appropriate uncertainty handling when terminology differed from transcript content. No hallucinations or incorrect substitutions occurred. Readability analysis showed a mean Flesch-Kincaid Grade Level of 8.56 and a Reading Ease score of 60.87, indicating accessible patient-level language. CONCLUSIONS: A RAG conversational agent grounded in consultation transcripts can deliver highly accurate, patient-friendly recall support, with strong factual reliability and appropriate uncertainty handling. Temporal integration remains the primary area for improvement. These findings demonstrate the feasibility of transcript-based patient assistance and support future development toward real-world deployment.

Ähnliche Arbeiten