Effect of AI-assisted software on inter- and intra-observer variability for preschool children X-ray bone age assessment

Abstract

Abstract Background With the rapid development of deep learning algorithms and the rapid improvement of computer hardware in the past few years, AI-assisted diagnosis software for bone age has achieved good diagnostic performance. The purpose of this study was to investigate the effect of AI-assisted software on residents’ inter-observer agreement and intra-observer reproducibility for preschool children X-ray bone age assessment. Methods This prospective study was approved by Institutional Ethics Committee. Six board-certified residents interpreted 56 bone age radiographs ranging from 3 to 6 years with structured reporting by modified TW3 method. The images were interpreted on two separate occasions, once with and once without the assistant of AI. After a washout period of 4 weeks, the radiographs were reevaluated by each resident in the same way. The average bone age results of three experts were the reference bone age. Both TW3-RUS and TW3-Carpal were evaluated. The root mean squared error (RMSE), mean absolute difference (MAD) and bone age accuracy within 0.5 years & 1 year were used as metrics of accuracy. Inter-observer agreement and intra-observer reproducibility were evaluated using intraclass correlation coefficient (ICCs). Results With the assistance of bone age AI software, the accuracy of residents’ results improved significantly. For Inter-observer agreement comparison, The ICC results with AI assistance among 6 residents were higher than results without AI assistance on the two separate occasions. For intra-observer reproducibility comparison, the ICC results with AI assistance were higher than results without AI assistance between the 1st reading and 2nd reading for each resident. Conclusions For preschool children X-ray bone age assessment, besides improving diagnostic accuracy, bone age AI-assisted software can also increase inter-observer agreement and intra-observer reproducibility. AI-assisted software can be an effective diagnostic tool for residents in actual clinical settings.

Ähnliche Arbeiten