Bringing artificial intelligence safely to the clinics: hope is not a strategy

Abstract

The rate of development of new artificial intelligence (AI) models has surged dramatically in recent times; however, within health care, implementation of these new AI models is slow, with many models stalling at development and validation stage.1 While there may be several reasons for this so-called “implementation gap,”1 one proposed reason is a lack of safety (perceived or actual) of the models for clinical use.1 If AI models are to safely assist in important clinical decision-making, potentially providing clinical benefits while also minimizing risks to patients, it is imperative that we ensure that the AI is, in fact, performing well at all times. While it may be tempting to assume that a model, once trained and validated, will continue to perform effectively, we know that AI models decline in performance over time.2 Thus, any institution looking to implement a clinical AI model should also implement adequate quality control of that model. An obvious solution to the risk of declining performance is to continuously monitor the performance of clinical AI models, thereby ensuring safe operation of the model throughout its lifetime. However, simple as this may sound, monitoring clinical AI in practice is not simple at all. There are a multitude of methods to employ, the choice of which will depend on the clinical context, and will include balancing ethical, economical, practical, and statistical concerns. In many cases, the direct monitoring of model performance is not a possibility, and indirect forms of evaluating model performance may need to be considered instead.3 At our institution, we wished to implement AI solutions in the clinic and, thus, needed to carefully consider which monitoring schemes to use. As we began looking into this topic, we found almost no practical guidance on how to choose a monitoring strategy or examples of real-life implementation to rely on, and we found no summaries of available monitoring strategies. To remedy this, and to provide a starting point for not only ourselves but also for others undertaking similar projects, we decided to produce an overview of the monitoring methods currently proposed or used for AI in clinics. To get a comprehensive overview of the topic, we decided to include as broad a selection of sources of evidence as possible. We selected the scoping review methodology, which is a useful methodology when the sources are complex and heterogeneous.4 This led us to screen more than 13,000 sources, ultimately including and summarizing 39 sources, the results of which are published in this issue of JBI Evidence Synthesis.5 Our review provides a summary of options to monitor the use of AI in the clinical context, which both researchers and institutions looking to implement AI may use to plan their future work. Additionally, considering the enormous interest in AI in health care in general and the thousands of sources covering the development and validation of clinical AI, the limited 39 sources we identified as dealing concretely with monitoring clinical AI indicate a noticeable lack of focus on this important topic. Specifically, it is concerning that very few sources report having implemented any form of AI monitoring, and that we did not identify any guidelines that might help institutions in choosing and implementing sufficient AI monitoring. If an AI model is implemented without a coherent strategy for monitoring performance, then there are no guarantees that the AI will stay safe and efficient throughout its lifespan. The lack of focus on clinical AI monitoring is, therefore, quite alarming, considering the potentially detrimental effects that malfunctioning clinical AI may have. Further, as implied earlier in this editorial, lack of efficient quality control may be hampering the introduction of AI in clinics altogether. Merely hoping that clinical AI will keep working as intended is plainly not enough, and there is clearly a need for increased attention from researchers, governmental bodies, and implementing institutions to help develop and disseminate viable monitoring strategies for clinical AI. Hope is no such strategy.

Ähnliche Arbeiten