RAG Chatbot Architecture for Law & Crime News Using Hybrid Retrieval and Small Language Model

Abstract

The current massive development of RAG-based chatbots demonstrates the critical importance of personalization and response quality in language models. The personalization process in RAG lies in how its architecture processes dataset documents and matches them with user queries. In the response generation process, Large Language Models (lLMs) are frequently used as response generators in RAG-based chatbots. LLMs have very large sizes and parameters, requiring high computational costs. Several recent studies have shown that RAG-based chatbots with Small Language Models (SLMs) could compete with systems based on architectures equipped with LLMs. Our proposed architecture mitigates this dependency on LLMs by utilizing Gemma2:2B-instruct as an SLM. However, the efficacy of SLM is directly correlated with the quality of their input context. To satisfy this strict requirement, we utilize hybrid retrieval (BM25 and FAISS) and a reranking process that is designed to filter retrieval noise and accurately isolate the most relevant passages from the candidate pool. These passages are then employed by the SLM to synthesize the final answer. We measured the retrieval and response results with law and crime news articles using the classic ROUGE metric and the RAG-specific metric, RAGAs. The evaluation scores demonstrated that our RAG-based chatbot architecture performed very well in retrieval with 0.98 in Context Precision and 0.96 in Context Recall and moderately response generation with 0.82 in ROUGE-1. 0.7 in ROUGE-2, 0.78 in ROUGE-L, and 0.75 in Answer Correctness.

Ähnliche Arbeiten