Generative AI-Based Privacy-Preserving Next-Gen Student Support and Mental Health Navigation System

Abstract

The increasing demand for accessible, 24/7 student support services in higher education presents a significant challenge, particularly in the domain of mental health. Generative AI and Large Language Models (LLMs) offer a scalable solution, yet their deployment is fraught with a critical dilemma between the performance and safety of proprietary models and the data privacy afforded by open-source alternatives. This paper presents a comparative study of four leading LLMs (open-source Llama 3 70B and DeepSeek R1; proprietary GPT-4o Mini and Gemini 2.5 Flash) for a university-specific student support chatbot at Thompson Rivers University. Using a custom knowledge base and a novel six-dimensional evaluation framework (Factual Accuracy, Contextual Relevance, Completeness, Tone & Empathy, Practical Utility, and Safety & Risk Management), we analyzed over 200 model responses to 50+ real-world student queries. The results reveal that while open-source models demonstrate parity in factual accuracy for academic questions, a critical performance gap exists in the human-centric attributes of safety, empathy, and utility, particularly in response to sensitive mental health queries. The study concludes that current open-source models, in their base form, are not yet suitable for unmonitored, student-facing deployment in high-stakes scenarios. We propose a hybrid deployment model as a pragmatic and responsible path forward for universities seeking to balance innovation with student welfare and data sovereignty.

Ähnliche Arbeiten