Effectively Crowdsourcing Radiology Report Annotations.

Abstract

Crowdsourcing platforms are a popular choice for researchers to gather text an-notations quickly at scale. We inves-tigate whether crowdsourced annotations are useful when the labeling task requires medical domain knowledge. Comparing a sentence classification model trained with expert-annotated sentences to the same model trained on crowd-labeled sentences, we find the crowdsourced training data to be just as effective as the manually pro-duced dataset. We can improve the ac-curacy of the crowd-fueled model with-out collecting further labels by filtering out worker labels applied with low confidence. 1

Ähnliche Arbeiten