Length-of-Stay Prediction with Data Fusion and Masked Language Modeling

Abstract

Length of Stay (LOS) prediction plays a pivotal role in efficient healthcare resource management, optimization of care quality, and reduction of treatment costs. One of the key challenges in this domain is the complexity of processing unstructured long clinical notes containing sensitive information. Moreover, limitations persist in effectively processing long texts and integrating multiple information sources. This paper therefore proposes a method integrating two medical data sources: structured demographic and admission data, unstructured long clinical notes. Both can be then transformed into a unified text input. This data combination simultaneously leverages quantitative information from tabular data (demographic and initial admission information) and rich supplementary information from clinical notes (medical history, detailed symptoms, etc.), providing a more comprehensive picture of patient condition and improving prediction performance. Additionally, the paper proposes applying masked language modeling to enhance the ModernBERT model to create more effective general representations of entire patient texts compared to traditional approaches that rely solely on BERT. Evaluated on MIMIC-III data, our resulting Ad-ModernBERT model can yield better LOS predictions consistently in many experiments.

Ähnliche Arbeiten