Strengths and Limitations of Machine Learning in Surgical Care

Abstract

Bertsimas and co-authors 1 should be commended on the development of the Predictive Optimal Trees in Emergency Surgery Risk (POTTER) tool, and its most recent application in emergency general surgery. 2 The use of machine learning-based risk prediction tools as the basis for this study opens the door to a timely discussion about the role of machine learning approaches in support of decision-making in surgery.The following critical concepts must be explored before the adoption of POTTER and similar models for use in clinical settings: strengths and limitations of machine learning methods, including mechanisms to fairly assess their performance; and the importance of inclusion of hospital or surgeon factors when modeling.Machine learning-based tools can outperform traditional modeling methods, such as logistic regression. 3-5However, machine learning methods have limitations that can impact their utility in clinical use.For example, machine learning methods often lack the ability to provide principled measures of uncertainty around the predictions.Many machine learning methods do not rely on statistical models, unlike regression, to generate predictions.As a consequence, these machine learning-based predictions lack CIs, SEs, or other interpretable metrics of uncertainty quantification.This makes it difficult for surgeons to put the predictions into context for patients who need more than percentages to inform decisions.In this case, the POTTER model relies on decision trees, a classical machine learning tool that does not come with uncertainty quantification, as evidenced by Tables 2 and3. 2 In this context, SEs or other measures of uncertainty would be useful to surgeons, for instance, to distinguish a numerical improvement from one with a certain chance of being clinically relevant.On another note, POTTER has only been evaluated on datasets of real patients.Evaluating POTTER on datasets of simulated patients would also be beneficial.The evaluation of machine learning methods on simulated data, where uncertainty can be controlled and modulated according to a specific design, is useful to form expectations about model performance in an idealized setting that mirrors the target patient population in important ways.In particular, simulations allow developers to probe and identify clinical scenarios or patient cohorts on which the model can be expected to perform poorly.When further examining the POTTER tool, we noted that the model lacks information on the site of care.Although this is not unique to POTTER, the inability to factor in hospital-level

Ähnliche Arbeiten