Labelling imaging datasets on the basis of neuroradiology reports: a\n validation study

Abstract

Natural language processing (NLP) shows promise as a means to automate the\nlabelling of hospital-scale neuroradiology magnetic resonance imaging (MRI)\ndatasets for computer vision applications. To date, however, there has been no\nthorough investigation into the validity of this approach, including\ndetermining the accuracy of report labels compared to image labels as well as\nexamining the performance of non-specialist labellers. In this work, we draw on\nthe experience of a team of neuroradiologists who labelled over 5000 MRI\nneuroradiology reports as part of a project to build a dedicated deep\nlearning-based neuroradiology report classifier. We show that, in our\nexperience, assigning binary labels (i.e. normal vs abnormal) to images from\nreports alone is highly accurate. In contrast to the binary labels, however,\nthe accuracy of more granular labelling is dependent on the category, and we\nhighlight reasons for this discrepancy. We also show that downstream model\nperformance is reduced when labelling of training reports is performed by a\nnon-specialist. To allow other researchers to accelerate their research, we\nmake our refined abnormality definitions and labelling rules available, as well\nas our easy-to-use radiology report labelling app which helps streamline this\nprocess.\n

Ähnliche Arbeiten