Dies ist eine Übersichtsseite mit Metadaten zu dieser wissenschaftlichen Arbeit. Der vollständige Artikel ist beim Verlag verfügbar.
Assessment of the Capacity of ChatGPT as a Self-Learning Tool in Medical Pharmacology: A Study Using MCQs
3
Zitationen
1
Autoren
2023
Jahr
Abstract
Abstract Background ChatGPT is a large language model (LLM) developed by OpenAI. Drawing upon a substantial textual corpus, ChatGPT exhibits a remarkable ability to simulate human speech. This investigation attempts to evaluate the potential of ChatGPT as a self-directed learning resource, with specific attention to its efficacy in answering multiple-choice questions (MCQs) and furnishing coherent rationale for its responses. Methods The study used 78 MCQs from the Korean Basic Medical Science Comprehensive Assessment (K-BMSCA) for years 2019 to 2021. The test items were translated from Korean to English, and lead-in prompts were engineered to test ChatGPT's ability to generate correct and relevant responses. Each test item produced four questions with different prompts. The questions were submitted to ChatGPT, and the responses were analyzed for correctness, consistency, and relevance. Results The study evaluated the efficacy of ChatGPT in responding to a total of 312 questions with an overall accuracy of 76.0% (237/312). The model demonstrated a commendable performance in addressing questions related to recall and interpretation, whereas its performance in solving problems was comparatively poor. ChatGPT offered correct rationales for 77.8% (182/234) of the responses, with errors primarily arising from two sources: faulty information and flawed reasoning. In terms of references, ChatGPT furnished incorrect citations for 69.7% (191/274) of the responses. While the veracity of reference paragraphs could not be ascertained, 80.4% (41/51) were deemed pertinent and accurate with respect to the answer key. Conclusion While ChatGPT has potential utilities in answering MCQs and generating the correct associated rationale, it shows poor performance in referencing. The limitations of ChatGPT as a self-learning tool should be explained and emphasized to students for a wise usage of AI technology.
Ähnliche Arbeiten
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
2019 · 8.245 Zit.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
2019 · 8.102 Zit.
High-performance medicine: the convergence of human and artificial intelligence
2018 · 7.468 Zit.
Proceedings of the 19th International Joint Conference on Artificial Intelligence
2005 · 5.776 Zit.
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
2018 · 5.429 Zit.