Predicting Discharge Disposition Following Meningioma Resection Using a Multi-Institutional Natural Language Processing Model

Abstract

BACKGROUND: Machine learning (ML)-based predictive models are increasingly common in neurosurgery, but typically require large databases of discrete variables for training. Natural language processing (NLP) can extract meaningful data from unstructured text. OBJECTIVE: To present an NLP model that predicts nonhome discharge and a point-of-care implementation. METHODS: We retrospectively collected age, preoperative notes, and radiology reports from 595 adults who underwent meningioma resection in an academic center from 1995 to 2015. A total of 32 algorithms were trained with the data; the 3 best performing algorithms were combined to form an ensemble. Predictive ability, assessed by area under the receiver operating characteristic curve (AUC) and calibration, was compared to a previously published model utilizing 52 neurosurgeon-selected variables. We then built a multi-institutional model by incorporating notes from 693 patients at another center into algorithm training. Permutation importance was used to analyze the relative importance of each input to model performance. Word clouds and non-negative matrix factorization were used to analyze predictive features of text. RESULTS: The single-institution NLP model predicted nonhome discharge with AUC of 0.80 (95% CI = 0.74-0.86) on internal and 0.76 on holdout validation compared to AUC of 0.77 (95% CI = 0.73-0.81) and 0.74 for the 52-variable ensemble. The multi-institutional model performed similarly well with AUC = 0.78 (95% CI = 0.74-0.81) on internal and 0.76 on holdout validation. Preoperative notes most influenced predictions. The model is available at http://nlp-home.insds.org. CONCLUSION: ML and NLP are underutilized in neurosurgery. Here, we construct a multi-institutional NLP model that predicts nonhome discharge.

Ähnliche Arbeiten