Graph-Based Medical Validation of Black-Box AI Clinical Decision Support

Abstract

The increasing adoption of black-box AI systems-LLMs in particular-in Clinical Decision Support Systems (CDSSs) has raised concerns regarding the transparency and explainability of their recommendations. This paper introduces GraVAL, a graph-based approach designed to validate the suggestions generated by blackbox CDSS. By leveraging the UMLS and other medical ontologies, GraVAL constructs a knowledge graph that captures medical entities and their interrelations. A semantic distance measure is defined to assess the relevance of recommendations, which are validated through a retrospective analysis. We evaluate the effectiveness of GraVAL using a benchmark dataset derived from the MIMIC-IV clinical database, demonstrating that the approach successfully validates up to 95% of CDSS recommendations. Additionally, the resulting shortest paths provide transparent, interpretable explanations of why specific recommendations are relevant, thus aligning with emerging regulations for trustworthy AI in healthcare. Our results suggest that GraVAL can substantially enhance the reliability and accountability of AI-driven decision support tools in clinical settings.

Ähnliche Arbeiten