"AI that’s so good it almost seems human:" Risk and Hype Cycles of AI and Chatbots in Mobile Mental Health Apps

Abstract

As interactive AI becomes increasingly integrated into digital mental health tools, understanding trends in this innovation landscape is critical. This study examines developer claims and user experiences with mental health apps featuring conversational agents (CA) and other interactive AI features. Through a critical discourse analysis of app descriptions and user reviews we found that consumer experiences with interactive AI and CA for mental health are complicated by a lack of regulation, placing the onus on users to manage safety risks. Interactive AI features within apps created to aid mental health often claim efficacy without adequate scientific support, and can worsen mental health conditions due to lack of maturity in AI algorithms embedded in CAs, causing harm to those struggling with mental health. We therefore advocate for safety-focused design, guided by principles of Safety by Design, alongside stricter regulations to ensure these tools are responsibly developed and centred on user needs.

Ähnliche Arbeiten