Evaluating Generative Artificial Intelligence–Drafted Responses for Patient Messages in Obstetrics and Gynecology

Abstract

OBJECTIVE: To assess the utility of generative artificial intelligence (AI) draft responses for patient messages in relation to clinician workload, burnout, and perceptions of messaging efficiency and quality in an obstetrics and gynecology practice. METHODS: We conducted a prospective study among physicians and nonphysician clinicians at a large academic hospital across three outpatient sites, integrating GPT-4 into the electronic health record (EHR) to generate patient portal draft responses. Drafts appeared in the clinician's inbox, with options to edit or disregard. Primary outcomes included changes in perceived workload, measured using the Physician NASA Task Load Index, and burnout, measured using the Mini Z 2.0 survey. Secondary outcomes included draft utilization rates and changes in clinician-reported attitudes toward generative AI-assisted messaging. Only survey responders were included in the study. Generative AI usage data were captured over a 3-month intervention period, and preintervention and postintervention survey responses were analyzed using descriptive statistics and nonparametric tests. RESULTS: Twenty physicians (69.0%) and nine nonphysician clinicians (31.0%) were included. Among the physicians and nonphysician clinicians, 86.2% of participants were female. The largest age group represented was 35–44 years (37.9%, 11/29); 69.0% participants identified as White, and 37.9% had more than 20 years of posttraining clinical experience. Seventy-five percent of participants reported at least some reduction in stress or burnout related to managing patient portal messages. There were 1,700 AI-generated draft responses produced, with an overall draft usage rate of 16.2%. Physicians demonstrated higher usage (28.6%) than nonphysician clinicians (13.0%). Mean (SD) perceived workload scores decreased from 221.21 (71.54) to 187.90 (80.21) postintervention ( P =.041). Physicians reported a significant reduction in workload (mean [SD] 208.65 [70.67] to 166.85 [59.81], P =.039), whereas nonphysician clinicians did not (249.11 [69.15] to 234.67 [102.24], P =.61). Mean (SD) Mini Z burnout scores improved from 28.38 (4.96) to 30.76 (5.69) ( P <.001). Postintervention survey responses indicated moderate perceived improvements in messaging efficiency and quality. CONCLUSION: Generative AI draft integration in obstetrics and gynecology messaging was associated with reduced EHR workload scores, modest burnout reduction, and moderate satisfaction with message efficiency and quality. Specialty-tuned generative AI tools may be more useful for improving inbox management.

Ähnliche Arbeiten