Prospective Evaluation of Large Language Model Integration into a Classical Hematology Case Conference (Preprint)

Abstract

<sec> <title>BACKGROUND</title> Large-language models (LLMs) have emerging applications in clinical decision-making and medical education, but prospective evaluations in hematology are limited. </sec> <sec> <title>OBJECTIVE</title> We conducted a prospective feasibility study examining the integration of two LLM-based models into a weekly classical hematology case conference. </sec> <sec> <title>METHODS</title> Over 8 consecutive sessions, ChatGPT and Open Evidence AI were incorporated into real-time case discussions. Presenters used structured prompts to obtain differential diagnoses, diagnostic pathways, guideline-supported management options, and reference retrieval. AI outputs were displayed during the conference and discussed alongside clinical reasoning by hematology faculty. After the intervention, 25 attendees completed a structured survey assessing changes in familiarity and use of AI, perceived value, observed limitations, and preferred implementation strategies. </sec> <sec> <title>RESULTS</title> Participants included 16 attending hematologists (64%) and 7 trainees (28%). Familiarity with AI increased from 16% “very familiar” prior to the intervention to 36% “a lot of familiarity” afterward. Frequent or occasional AI use increased from 44% to 68%. Most respondents (84%) rated AI as “very” or “somewhat valuable.” AI was most often perceived as helpful for suggesting alternative diagnoses (80%) and retrieving relevant references (92%). Limitations included prompt dependency (60%), insufficient personalization (52%), and occasional irrelevant or incomplete recommendations (52%). Nearly all respondents (92%) favored an adjunctive rather than self-supervised role for AI. </sec> <sec> <title>CONCLUSIONS</title> Prospective integration of LLM tools into a classical hematology challenging cases conference was feasible, increased clinician familiarity and interest, and was perceived as diagnostically and educationally valuable. Future investigations should evaluate accuracy, reliability, and optimal frameworks for structured, supervised AI use in hematology education. </sec> <sec> <title>CLINICALTRIAL</title> Not applicable </sec>

Ähnliche Arbeiten