Improving the robustness and accuracy of biomedical language models\n through adversarial training

Abstract

Deep transformer neural network models have improved the predictive accuracy\nof intelligent text processing systems in the biomedical domain. They have\nobtained state-of-the-art performance scores on a wide variety of biomedical\nand clinical Natural Language Processing (NLP) benchmarks. However, the\nrobustness and reliability of these models has been less explored so far.\nNeural NLP models can be easily fooled by adversarial samples, i.e. minor\nchanges to input that preserve the meaning and understandability of the text\nbut force the NLP system to make erroneous decisions. This raises serious\nconcerns about the security and trust-worthiness of biomedical NLP systems,\nespecially when they are intended to be deployed in real-world use cases. We\ninvestigated the robustness of several transformer neural language models, i.e.\nBioBERT, SciBERT, BioMed-RoBERTa, and Bio-ClinicalBERT, on a wide range of\nbiomedical and clinical text processing tasks. We implemented various\nadversarial attack methods to test the NLP systems in different attack\nscenarios. Experimental results showed that the biomedical NLP models are\nsensitive to adversarial samples; their performance dropped in average by 21\nand 18.9 absolute percent on character-level and word-level adversarial noise,\nrespectively. Conducting extensive adversarial training experiments, we\nfine-tuned the NLP models on a mixture of clean samples and adversarial inputs.\nResults showed that adversarial training is an effective defense mechanism\nagainst adversarial noise; the models robustness improved in average by 11.3\nabsolute percent. In addition, the models performance on clean data increased\nin average by 2.4 absolute present, demonstrating that adversarial training can\nboost generalization abilities of biomedical NLP systems.\n

Ähnliche Arbeiten