Can AI Models like ChatGPT and Gemini Dispel Myths About Children’s and Adolescents’ Mental Health? A Comparative Brief Report

Abstract

Background: Dispelling myths is crucial for policy and health communication because misinformation can directly influence public behavior, undermine trust in institutions, and lead to harmful outcomes. This study aims to assess the effectiveness and differences between OpenAI’s ChatGPT and Google Gemini in dispelling myths about children’s and adolescents’ mental health. Methods: Using seven myths about mental health from the UNICEF & WHO Teacher’s Guide, ChatGPT-4o and Gemini were asked to “classify each sentence as a myth or a fact”. Responses of each LLM for word count, understandability, readability and accuracy were analyzed. Results: Both ChatGPT and Gemini correctly identified all 7 statements as myths. The average word count of ChatGPT’s responses was 60 ± 11 words, while Gemini’s responses averaged 60 ± 29 words, a statistically non-significant difference between the LLMs. The Flesch–Kincaid Grade Level averaged 11.7 ± 2.2 for ChatGPT and 10.2 ± 1.3 for Gemini, also a statistically non-significant difference. In terms of readability, both ChatGPT and Gemini’s answers were considered difficult to read, with all grades exceeding the 7th grade level. The findings should nonetheless be interpreted with caution due to the limited dataset. Conclusions: The study adds valuable insights into the strengths of ChatGPT and Gemini as helpful resources for people seeking medical information about children’s and adolescents’ mental health, although the content may not be as easily accessible to those below a college reading level.

Ähnliche Arbeiten