RMUTTOBot: Transforming University Admission Services with a TAG-based RAG LLM Chatbot

Abstract

Advancements in artificial intelligence, particularly in large language models (LLMs) and retrieval-augmented generation (RAG) techniques, have improved chatbot capabilities for more natural and domain-specific interactions. However, conventional RAG systems, which retrieve information from unstructured text sources like websites and PDFs, exhibit critical failures when applied to the dynamic and precise nature of university information. This research addresses these gaps through the design and development of RMUTTOBot, a domain-specific chatbot providing admissions support for prospective students at Rajamangala University of Technology Tawan-ok (RMUTTO). We propose a novel, lightweight table-augmented generation (TAG) approach that combines a curated, updatable knowledge base for general information with live database queries for real-time, dynamic data. Performance was evaluated using both automated metrics and human assessments across six criteria: semantic similarity, retrieval effectiveness, relevance, fluency, coverage, and consistency. Experimental results show that the TAG-based RAG system significantly outperformed both the baseline LLM-only configuration and PDF-based RAG system, achieving a 12.76% higher BERTF1 score compared to a PDF-based RAG. Human evaluation confirmed the system’s high response relevance and linguistic fluency, with strong inter-rater reliability (Krippendorff’s α > 0.835). These findings demonstrate that combining structured data augmentation with RAG substantially enhances chatbot accuracy, contextual grounding, and completeness, offering a robust framework for intelligent conversational systems in academic domains. The source code and implementation details are publicly available at https://github.com/vipa-thananant/RMUTTOBot.

Ähnliche Arbeiten