Comparative Impact of ChatGPT and Conventional Search Tools on Clinical Reasoning Performance: A Randomized Crossover Study in Preclinical Medical Students

Abstract

Purpose: To compare the impact of ChatGPT and conventional search strategies on clinical reasoning performance among preclinical medical students. Patients and Methods: A randomized crossover study was conducted at a single institution during a musculoskeletal system course involving 46 second-year medical students. Participants completed a baseline pre-test followed by two structured intervention phases in which they analyzed standardized clinical cases using either ChatGPT or conventional search tools (Google or PubMed). A 60-minute washout period was implemented before crossover to the alternate modality. Post-tests were administered after each phase. The primary outcome was clinical reasoning performance measured using an eight-point rubric-based scale. Secondary outcomes included self-perceived learning, confidence, and qualitative feedback. Paired t-tests were used for within-subject comparisons, and effect sizes were calculated using Cohen's d. Results: The mean pre-test score was 3.96 (standard deviation 1.65), increasing to 4.96 (standard deviation 1.71) after the first intervention and to 5.70 (standard deviation 1.50) after crossover. Improvements were statistically significant across all paired comparisons (p < 0.05), with a large cumulative effect size (Cohen's d = 0.97). Performance improvements were observed across both learning modalities, without evidence that gains were attributable to a single approach. Students reported that ChatGPT facilitated rapid organization of differential diagnoses and management plans, whereas conventional search encouraged more deliberate synthesis and comparison of information sources. Conclusion: Both artificial intelligence-assisted and conventional search strategies improved short-term clinical reasoning performance within a structured active-learning environment. These findings support a balanced integration of large language models alongside traditional search methods in undergraduate medical education. Clinical Trial Registration Number: TCTR20260218005.

Ähnliche Arbeiten