Statistical Analyses and Reporting of Performance of AI Models

Abstract

Statistics are essential for understanding, developing and evaluating applications of artificial intelligence (AI) in health care. This chapter will first cover key statistical concepts, such as data types, inference and bias, before exploring the predictive statistics most used in AI. The chapter then focuses on how to assess the performance of a clinical AI model. How often do we read a paper asserting that the performance of an AI model exceeds that of doctors? But what does that mean? Perhaps, our algorithm identifies more cases of cancer successfully, but at the cost of many more false positives and, thus, unnecessary further testing. Another challenge emerging from this fusion of AI and statistics is nomenclature. As clinicians, we may have heard of sensitivity and specificity. But what about precision and recall which data scientists use? When should I use one or the other? Ultimately the objective of this chapter is to cover commonly used statistical methods in medical AI research needed to critically evaluate the relevant literature. Throughout practical examples and AI studies that evaluate models using the different statistical methods are described. At the end of the chapter, you will find further reading.

Ähnliche Arbeiten