AI Ethics Statements -- Analysis and lessons learnt from NeurIPS Broader\n Impact Statements

Abstract

Ethics statements have been proposed as a mechanism to increase transparency\nand promote reflection on the societal impacts of published research. In 2020,\nthe machine learning (ML) conference NeurIPS broke new ground by requiring that\nall papers include a broader impact statement. This requirement was removed in\n2021, in favour of a checklist approach. The 2020 statements therefore provide\na unique opportunity to learn from the broader impact experiment: to\ninvestigate the benefits and challenges of this and similar governance\nmechanisms, as well as providing an insight into how ML researchers think about\nthe societal impacts of their own work. Such learning is needed as NeurIPS and\nother venues continue to question and adapt their policies. To enable this, we\nhave created a dataset containing the impact statements from all NeurIPS 2020\npapers, along with additional information such as affiliation type, location\nand subject area, and a simple visualisation tool for exploration. We also\nprovide an initial quantitative analysis of the dataset, covering\nrepresentation, engagement, common themes, and willingness to discuss potential\nharms alongside benefits. We investigate how these vary by geography,\naffiliation type and subject area. Drawing on these findings, we discuss the\npotential benefits and negative outcomes of ethics statement requirements, and\ntheir possible causes and associated challenges. These lead us to several\nlessons to be learnt from the 2020 requirement: (i) the importance of creating\nthe right incentives, (ii) the need for clear expectations and guidance, and\n(iii) the importance of transparency and constructive deliberation. We\nencourage other researchers to use our dataset to provide additional analysis,\nto further our understanding of how researchers responded to this requirement,\nand to investigate the benefits and challenges of this and related mechanisms.\n

Ähnliche Arbeiten