Roadmap of Designing Cognitive Metrics for Explainable Artificial\n Intelligence (XAI)

Abstract

More recently, Explainable Artificial Intelligence (XAI) research has shifted\nto focus on a more pragmatic or naturalistic account of understanding, that is,\nwhether the stakeholders understand the explanation. This point is especially\nimportant for research on evaluation methods for XAI systems. Thus, another\ndirection where XAI research can benefit significantly from cognitive science\nand psychology research is ways to measure understanding of users, responses\nand attitudes. These measures can be used to quantify explanation quality and\nas feedback to the XAI system to improve the explanations. The current report\naims to propose suitable metrics for evaluating XAI systems from the\nperspective of the cognitive states and processes of stakeholders. We elaborate\non 7 dimensions, i.e., goodness, satisfaction, user understanding, curiosity &\nengagement, trust & reliance, controllability & interactivity, and learning\ncurve & productivity, together with the recommended subjective and objective\npsychological measures. We then provide more details about how we can use the\nrecommended measures to evaluate a visual classification XAI system according\nto the recommended cognitive metrics.\n

Ähnliche Arbeiten