Turning Off Your Better Judgment – Conformity to Algorithmic Recommendations

Abstract

In many practical settings, humans rely on artificial intelligence algorithms to facilitate decision processes: from choosing driving routes and picking leisure activities to defining investment plans and guiding judicial procedures. Given that algorithms are known to be vulnerable to systematic errors and biases, this research seeks to understand: To what degree do humans take algorithmic advice at face value, even when such advice might be expected to conflict with their better judgment? To address this question, we draw from classic studies of social conformity and explore the extent to which gig-economy workers engaged in simple and objective perceptual-judgment (image classification) tasks conform to algorithmic recommendations that are clearly erroneous. Results from three studies (n = 1,085) show that substantial percentages of workers follow erroneous algorithmic recommendations (on 8.8%–26.5% of tasks); workers who do not view such recommendations do not make similar mistakes independently (only on about 1% of tasks). Moreover, workers are more likely to conform to erroneous algorithmic recommendations than to identical recommendations generated by other humans. We further show that workers become less likely to conform to an erroneous algorithmic recommendation when it is presented alongside a second, conflicting (correct) recommendation. Finally, conformity diminishes when workers perceive the real-life impact of their decisions as high (versus low). Given our realistic setup, our findings are directly applicable to workers engaged in hybrid machine–human judgment tasks, in addition to providing broader insights into the nature of human reliance on algorithms—and the risks it might entail.

Ähnliche Arbeiten