Language Models Lag Behind: Inconsistent Alignment with Recent Removal of Race from Clinical Algorithms

Abstract

<bold>BACKGROUND:</bold> Several medical societies have revised commonly used clinical algorithms to remove the social construct of race as a predictor. Given the rise in the adoption of large language models (LLMs), we assessed whether LLMs trained since October 2023 are aware of the recent removal of race from lung function reference equations and models used to estimate Glomerular Filtration Rate (eGFR). <bold>METHODS:</bold> We evaluated a combination of 12 state-of-the-art and lightweight language models by OpenAI, Meta, Microsoft, and DeepSeek. We used various prompts to ask LLMs to identify the latest FEV₁ and eGFR reference models and their input parameters, as recommended by the ERS/American Thoracic Society (ATS) and the American Society of Nephrology (ASN), respectively. We ran each prompt-model combination five times with default parameters and compared model outputs with the Medical Society's recommendations. <bold>RESULTS:</bold> Regardless of model size and reasoning capability, less than 3% of model outputs correctly identified the removal of race from lung function equations. In contrast, 100% of outputs from state-of-the-art models and 25% from lightweight models correctly identified the removal of race from eGFR (Figure). No statistically significant difference was observed between open-source and proprietary models. <bold>CONCLUSIONS:</bold> Evaluated LLMs consistently failed to correctly identify the latest recommendations of ERS and ATS. <fig><object-id>erj;66/suppl_69/PA2023/F1</object-id><object-id>F1</object-id><object-id>F1</object-id><graphic></graphic></fig>

Ähnliche Arbeiten