Refining AI’s role in colorectal cancer screening education: a reply addressing nuances of bias and comprehensive validation

Abstract

Dear Editor, We read with great interest the correspondence by Ji et al[1] regarding our recently published article, “Comparative analysis of artificial intelligence tools for the dissemination of colorectal cancer screening guidelines: a novel perspective on early screening education”[2]. We sincerely appreciate the insightful comments, which raise critical questions about the neutrality of Artificial Intelligence (AI), the practical utility of evaluation frameworks, and the inherent limitations of AI-based scoring systems. This thoughtful dialog is invaluable, as it underscores the multifaceted challenges and responsibilities inherent in the burgeoning field of AI integration into public health, particularly concerning the deployment of Large Language Models for critical medical information dissemination. First, regarding AI neutrality and bias, we agree that simplifying complex medical guidelines is not a neutral act; it inherently involves interpretation and omission, potentially introducing biases (demographic, socio-economic, and cultural)[3]. While our study focused on technical accuracy of guideline data points (e.g., screening age), we concur that the AI-generated output’s “tone,” “framing,” and assumptions are equally, if not more, critical. AI risks perpetuating outdated norms, stereotypes, or misrepresenting conditions. This reinforces our conclusion that AI, while a powerful initial drafting or information retrieval tool, requires strict “clinical validation and ethical oversight” by human experts before deployment. Future AI tools must be trained on diverse datasets and continuously monitored to minimize biases and ensure equitable information. Second, concerning prioritizing “form correctness” over “practical applicability,” we contend that for disseminating vital clinical guidelines, factual accuracy (i.e., “form correctness”) is an absolute prerequisite for utility. An AI tool, even if engaging, that cites incorrect screening ages or misinterprets high-risk criteria, as observed in our study, poses an unacceptable risk to patient safety. Therefore, establishing a robust baseline of factual and contextual accuracy is the essential first step. However, we agree that future studies must extend beyond this foundational assessment to incorporate patient-centric metrics, such as improved health literacy, enhanced patient empowerment for shared decision-making, and increased screening compliance and behavioral changes. Integrating user experience research and patient feedback will be crucial to bridge the gap between “form correctness” and true “practical applicability.” Third, regarding the limitation of AI-on-AI scoring, we recognize the inherent potential for circular reasoning and the risk of reinforcing existing model biases when one AI evaluates another. We utilized this method as a pragmatic, cost-effective, and standardized preliminary screening tool to efficiently handle large volumes of data – a methodology increasingly explored and refined in the computer science literature for initial performance benchmarking[4]. However, we fully accept and explicitly stated in our original article that AI evaluation can never replace the “Gold Standard” of a multidisciplinary human expert panel. As suggested, we are actively designing a follow-up study that will involve independent, blinded reviews by a diverse group of clinical oncologists, public health experts, and communication specialists. This human expert validation will serve to critically assess the AI-generated scores, identify nuanced errors, evaluate contextual appropriateness, and provide qualitative insights that AI models currently cannot capture. This human-centric approach forms the critical second phase of what we envision as a multi-phase validation framework for AI-generated health information. To further consolidate our discussion on the identified challenges, the commentators’ crucial insights, and our proposed directions for future enhancements for each AI tool and the overall framework, we present a summary in Table 1. Table 1 - Summary of AI tool performance and identified challenges in colorectal cancer screening guideline dissemination AI tool Key advantages Key challenges Commentator’s focus Future improvement directions ChatGPT-40 Strong ability to simplify complex information Outdated screening starting age; overly simplified high-risk protocols AI neutrality and bias; practicality of evaluation framework Regularly update training data; strengthen clinical validation, incorporate human expert review Claude 3.5 Provides a comprehensive framework Lacks critical implementation details Practicality of evaluation framework Refine output details, integrate with local health care systems, ensure operability DeepSeek Excellent regional adaptability and logical rigor Accuracy of thresholds needs improvement Practicality of evaluation framework; AI neutrality and bias Enhance accuracy of professional medical knowledge base; strengthen ethical review to avoid misleading information Overall (All AIs) Ability to translate complex guidelines into easily understandable language; increased coverage of early education Output content requires clinical validation and ethical oversight; lacks real user feedback Limitations of AI-on-AI review; underestimation of socio-ethical impact Introduce human expert validation; conduct user impact studies; develop ethical use guidelines; enhance multidisciplinary collaboration Finally, regarding the broader societal implications, we agree that delegating health education solely to AI carries significant risks, including fostering patient passivity, potentially reducing critical health literacy engagement, and exacerbating existing health inequalities through the digital divide. AI should optimally function as a sophisticated “co-pilot” or intelligent assistant, designed to enhance the efficiency and reach of medical professionals, rather than replacing the empathetic, nuanced communication, and ethical judgment that only a human doctor can provide[5]. The human element remains indispensable for building trust, understanding individual patient contexts, and delivering personalized care. Future research and development must therefore prioritize not only the accuracy but also the explainability and transparency of AI outputs, ensuring that patients and clinicians alike can understand the basis of the information provided and critically evaluate it. In conclusion, our initial study serves as a critical technical benchmark for current AI capabilities in a specific medical scenario. The thoughtful and constructive dialog initiated by Ji et al is invaluable. It profoundly highlights the urgent need for a holistic, multi-phased approach – combining rigorous technical evaluation with subsequent human expert validation, ethical scrutiny, and robust patient-centric impact assessment – to safely and effectively harness AI’s immense potential for cancer prevention and public health education. Ethics approval Not applicable. Consent Not applicable.

Ähnliche Arbeiten