Prediction of American Society of Anesthesiologists Physical Status Classification from Preoperative Clinical Text Narratives Using Natural Language Processing

Abstract

Abstract Importance Large volumes of unstructured text notes exist for patients in electronic health records (EHR) that describe their state of health. Natural language processing (NLP) can leverage this information for perioperative risk prediction. Objective Predict a modified American Society of Anesthesiologists Physical Status Classification (ASA-PS) score using preoperative note text, identify which model architecture and note sections are most useful, and interpret model predictions with Shapley values. Design Retrospective cohort analysis from an EHR. Setting Two-hospital integrated care system comprising a tertiary/quaternary academic medical center and a level 1 trauma center with a 5-state referral catchment area. Participants Patients undergoing procedures requiring anesthesia care spanning across all procedural specialties from January 1, 2016 to March 29, 2021 who were not assigned ASA VI and also had a preoperative evaluation note filed within 90 days prior to the procedure. Exposures Each procedural case paired with the most recent anesthesia preoperative evaluation note preceding the procedure. Main Outcomes and Measures Prediction of a modified ASA-PS from preoperative note text. We compared 4 different text classification models for 8 different input text snippets. Performance was compared using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Shapley values were used to explain model predictions. Results Final dataset includes 38566 patients undergoing 61503 procedures. Prevalence of ASA-PS was 8.81% for ASA I, 31.4% for ASA II, 43.25% for ASA III, and 16.54% for ASA IV-V. The best performing models were the BioClinicalBERT model on the truncated note task (macro-average AUROC 0.845) and the fastText model on the full note task (macro-average AUROC 0.865). Shapley values reveal human-interpretable model predictions. Conclusions and Relevance Text classification models can accurately predict a patient’s illness severity using only free-form text descriptions of patients without any manual data extraction. They can be an additional patient safety tool in the perioperative setting and reduce manual chart review for medical billing. Shapley feature attributions produce explanations that logically support model predictions and are understandable to clinicians.

Ähnliche Arbeiten