Letter to the Editor Regarding the Article: “Exploring the Capacities of <scp>ChatGPT</scp> : A Comprehensive Evaluation of Its Accuracy and Repeatability in Addressing <i>Helicobacter pylori</i> ‐Related Queries”

Abstract

We read with great interest the article entitled “Exploring the capacities of ChatGPT: A comprehensive evaluation of its accuracy and repeatability in addressing Helicobacter pylori-related queries” by Lai et al. [1]. The objective of this study was to evaluate the precision and reliability of chatGPT3.5 in addressing H. pylori-related queries. Two experts in the field of H. pylori independently assessed 21 H. pylori-related FAQs on topics including basic principles, diagnosis, treatment, and prevention. The results demonstrated that ChatGPT3.5 provided accurate and complete responses to 61.9% of the relevant questions, with the highest level of accuracy observed in the responses to questions pertaining to prevention. Additionally, the repeatability of ChatGPT3.5's answers to H. pylori-related questions reached a notable 95.23%. However, ChatGPT3.5 performed poorly in answering queries regarding H. pylori drug resistance and use of sensitive antibiotics. It is our intention to provide some insights and recommendations regarding this study. Firstly, researchers should utilize a standardized consultation checklist and employ three or more repetitions of questions in order to reduce the variability of responses generated by ChatGPT3.5, thereby improving the accuracy and reliability of responses to H. pylori-related questions [2]. Furthermore, the repeatability of ChatGPT3.5 responses should be assessed using a more scientific calculation method to prevent potential subjective decisions and limitations in the expert assessment process from affecting the scoring results [3, 4]. It is also recommended that future studies consider increasing the number of H. pylori-related questions. A smaller number of questions may not be able to cover and represent the entire population, which could limit the generalizability of the results. Furthermore, the study could investigate the discrepancies in accuracy, comprehensiveness, and capacity to furnish tailored information between ChatGPT3.5 responses to pertinent inquiries and those provided by professional medical practitioners, alternative information sources, or other AI language models. In conclusion, these findings demonstrate the significant potential of AI language models in providing medical information related to H. pylori. However, it is crucial to acknowledge the potential risks and limitations of relying solely on AI language models for medical information. We anticipate that further research and development in this area will contribute to the improvement of AI language models in terms of accuracy, reproducibility, and validity, ultimately benefiting individuals seeking medical knowledge. Conceptulization: Pingping Yang, Jiuliang Jiang. Validation: Pingping Yang, Jiuliang Jiang. Writing – original draft: Pingping Yang, Jiuliang Jiang. Writing – review and editing: Pingping Yang, Jiuliang Jiang. Supervision: Jiuliang Jiang. All authors read and approved the final manuscript. The authors have nothing to report. The authors have nothing to report. The authors declare no conflicts of interest.

Ähnliche Arbeiten