Chatbots based on artificial intelligence in oncourology: assessment of the reliability and quality of medical information.

Abstract

Introduction. In the era of digital progress and the development of artificial intelligence (AI) technologies, the potential application of chatbots in various fields, including medicine, has gained significant attention. With the global accessibility of the internet and the growing popularity of such services, there is an increasing need to evaluate the reliability and quality of the data generated, particularly concerning malignant neoplasms. The aim of this study was to analyze the quality of medical information related to prostate and bladder cancer using AI-based chatbots. Materials and methods. A study was conducted involving the examination and analysis of web traffic data from StatCounter. Based on the results of the web analysis to determine the leading search engine in overall web traffic in Russia, we used data from the Yandex Wordstat analytics service. We also conducted an assessment of responses from four AI-based chatbots to the most in-demand medical queries related to the two urological cancers with the highest incidence rates-prostate cancer and bladder cancer. We used publicly available versions of the four AI-based chatbots: ChatGPT, Perplexity, YaGPT, and GigaChat. To evaluate the quality of the medical information provided by the chatbots, we applied the validated DIS- CERN assessment tool. In addition, qualitative analysis and expert evaluation using DISCERN were carried out with the participation of 50 respondents-oncourologists with at least 10 years of professional experience in Russia. Results. We obtained and analyzed the evaluation results using the DISCERN scale. The highest overall score for medical information quality was achieved by the Perplexity chatbot, followed by ChatGPT in second place, and GigaChat in third. YaGPT ranked fourth and last. It is worth noting that all chatbots, to varying degrees, made errors and inaccuracies in generating medical information in response to the relevant queries. Conclusion. Today, AI-based chatbots represent a promising and in-demand direction. With the advancement of technology and the widespread adoption of software solutions, more people are turning to such services in search of authoritative medical information. However, at this stage, not all chatbots are capable of providing accurate medical content. Further development of AI architecture and internal algorithms is required, as well as the creation of standards and recommendations for integrating chatbots into medical information systems. This will help shape the direction of AI solutions for the coming decades and support both physicians and patients in diagnostics and decision-making.

Ähnliche Arbeiten