Subgroup performance and safety flags in pre‐operative machine learning

Abstract

Kotzé et al. present an ambitious translational programme that links routinely available primary care data with machine-learning models for ASA physical status and 30-day mortality, integrated directly into the ‘Smart PreOp’ system [1]. Their approach, which aligns model development with NHS workflow, deserves recognition and the reported discrimination in both development and external cohorts is impressive. From a clinical and public health perspective, this is a timely contribution to peri-operative digital innovation. I would, however, like to highlight two points that could be clarified using their data, which in turn would strengthen the practical value of their work. One issue is the real-world interpretability of ASA physical status. In the validation set at a decision threshold of 0.20, the authors report a precision of 0.95 and recall of 0.69 for patients who were ASA physical status 1–2, with reassuring decision-curve analysis. Yet approximately 105,794 (70%) of patients at Leeds Teaching Hospital and 69,437 (71%) in the Connected Bradford cohort were having elective surgery. Because clinical scheduling and capacity planning are organised and coded by admission type and surgical specialty, we would like to know the operating characteristics within these subgroups. Could the authors provide, per 1000 referrals, the confusion matrix counts and precision/recall at the 0.20 threshold and nearby cut-points, stratified by admission type and major coding groups, together with subgroup decision curves? This would not require new data collection but would allow services to translate model outputs into tangible terms, such as clinics saved or high-risk cases missed, within the units that drive elective pathway decisions. Another point concerns safety. The Methods section explains that the pipeline incorporates schema validation and a warning flag when inputs fall outside training ranges. This is an important safeguard, yet its actual behaviour has not been reported. It would be helpful to know how often the warning flag was triggered in both cohorts, what the common triggers were and whether flagged cases were associated with higher rates of ASA physical status misclassification or large mortality prediction errors. Such information, easily drawn from existing logs, would allow hospitals to plan escalation rules, such as mandatory senior review of flagged cases, and anticipate the workload involved in monitoring model reliability [2]. From a public health perspective, these additions are not just technical refinements. Peri-operative risk models do not exist in isolation from the wider diagnostic and laboratory ecosystem. Pathology services generate much of the data that underpin surgical decision-making and these same results are often the earliest indicators of physiological reserve. If machine-learning tools such as Smart PreOp are to be trusted, their outputs must align with, and ideally complement, the signals we see in routine laboratory practice. Subgroup-specific operating characteristics would allow managers to link risk stratification with patterns of laboratory demand, highlighting when streamlined pre-operative pathways might risk overlooking clinically significant abnormalities. Similarly, transparency about the warning flag system is critical. In pathology, we deal frequently with out-of-range values, and the way such anomalies are flagged or ignored can determine whether a result prompts urgent escalation or are dismissed. By connecting peri-operative machine learning models with the realities of laboratory medicine, the authors could help ensure that digital innovation strengthens both surgical pathways and diagnostic safety. Kotzé et al. provide an important demonstration of how machine learning can be embedded within NHS pre-operative assessment. Presenting subgroup-stratified performance and quantifying the warning flag system would enhance transparency, support safe adoption and broaden the impact on peri-operative services and health system planning.

Ähnliche Arbeiten