Evaluation of BERT and BART in Mental Health Dialogue by Using a Counselling Chatbot

Abstract

With the rising usage of AI for mental health support, counselling chatbots have gained significance in delivering counselling services. This research compares the performance of the BERT and BART models in constructing a counselling chatbot to assess their efficacy in generating meaningful responses. We employ the BERT model to obtain a response from the chatbot, utilizing its proficient contextual understanding capability. Conversely, we employed the BART model and Retrieval Augmented Generation (RAG) to enhance reaction speed by acquiring pertinent context from an external knowledge source. BLEU, ROUGE-I, ROUGE-2, ROUGE-L, and cosine similarity are among the metrics used to evaluate the models, along with answer accuracy, fluency, and contextual relevance. By obtaining a BLEU score of 0.49 and a ROUGE score of 0.71, the results show that the BART model is more effective than the BERT model in producing fluent responses.

Ähnliche Arbeiten