Groundedness in Government Document Chatbots: A Systematic Literature Review and Metric Oriented Analysis

Abstract

The application of Large Language Models (LLM) in public services encourages government agencies to adopt Retrieval Augmented Generation (RAG)-based chatbots as interfaces for regulatory knowledge and official documents. Although RAG is designed to increase the supportability of answers to authoritative sources, various studies show that this system is still vulnerable to hallucinations, which have the potential to reduce public trust and pose legal risks. This article presents a Systematic Literature Review (SLR) on the use of RAG in government chatbots with a focus on the definition, mitigation strategies, and evaluation of groundedness. The literature search was conducted in the period 2021–2025 through the SpringerLink, Scopus, and Taylor & Francis databases, resulting in 7,947 articles filtered using the PRISMA framework to obtain 100 articles Q1–Q2. Based on eight research questions, this study maps publication trends, document domains, RAG architecture, retrieval strategies, definitions of groundedness, and evaluation metrics used. The SLR results indicate conceptual fragmentation in the definition and measurement of groundedness, with the dominance of text-similarity-based metrics that are inadequate for regulatory contexts. As a conceptual contribution, this article formulates the Semantic Alignment Score (SAS) as a groundedness metric based on semantic alignment, evidence coverage, and entailment relationships, positioned to support the evaluation and auditing of government document chatbots.

Ähnliche Arbeiten