Evaluation of User Experience with RAG-based Chatbots for Searching Documentation: Industrial Case Study

Abstract

Artificial Intelligence (AI) has accelerated digital transformation across industries, with Large Language Models (LLMs) powering content generation, summarization, and dialogue systems, yet struggling with domain-specific knowledge. In industrial settings, Retrieval-Augmented Generation (RAG) architectures, often implemented as chatbots, address this issue by grounding responses in internal company knowledge. Despite increased industrial deployment, user experience evaluations of RAG-based chatbots remain limited, particularly regarding their effectiveness in supporting domain-specific workplace tasks. In this paper, we present a user evaluation of an RAG-based chatbot conducted in a medium-sized company. Employee feedback on chatbot usability and acceptance is analyzed to guide digitalization efforts in future AI-assisted enterprises.

Ähnliche Arbeiten