A Hybrid BERT–RAG Model for Developing Knowledge-Validated Conversational Systems

Abstract

The transition of freshmen into the university environment requires adaptive and responsive information support. This study develops a chatbot system based on a hybrid BERT–RAG architecture integrated with the FAISS Index to provide automated consultation services for new students. The novelty of this research lies in the implementation of a faculty-based hierarchical knowledge structure and an adaptive multi-domain context mechanism—an approach not previously found in studies involving BERT–RAG for university onboarding services. This design enables the chatbot to deliver more relevant, personalized, and faculty-specific responses. The dataset was derived from three primary sources of information: the Faculty of Economics and Business (FEB), the Faculty of Teacher Training and Education (FKIP), and the Faculty of Engineering (FT), which were structured into a validated knowledge base in documents.json format. System evaluation was conducted across ten interaction scenarios using performance metrics including BERT Similarity, BLEU Score, ROUGE-1, ROUGE-2, and ROUGE-L. The system achieved excellent results, with average scores of 0.905 (BERT Similarity), 0.844 (BLEU), 0.876 (ROUGE-1), 0.820 (ROUGE-2), and 0.871 (ROUGE-L) and standard deviations below 0.1 across all metrics. Strong metric correlations (0.85–0.99) further indicate consistency between semantic understanding and generated text quality. Furthermore, the system effectively minimizes hallucination through validated knowledge integration and faculty-based reranking strategies. Overall, this research provides a significant contribution to the development of institutionally contextual educational chatbots capable of delivering accurate, natural, and responsive communication to support new student orientation in higher education

Ähnliche Arbeiten