ChatGPT in the current form is not ready for unaudited use in healthcare and scientific research

Abstract

We read with interest the article on ChatGPT (OpenAI, San Francisco, California, USA) by Parikh et al.[1] and its accompanying editorial.[2] It was a well-thought-out article, and we have the following comments to make on this paper. The initial version of ChatGPT-3 was launched on November 30, 2022, and was found to be inaccurate and unreliable with regard to scientific publications.[3-5] Since then, OpenAI has introduced a newer version, i.e. ChatGPT-4, which is considered more reliable and accurate, with many additional features.[6] Because the survey by Parikh et al., reflected the sentiments of the participants about the older version of ChatGPT-3, the results may not have any relevance now. Furthermore, ChatGPT is an evolving technology, and the data generated will need to be accurate to apply to healthcare. This can be emphasized or reinforced only with rigorous data scrutiny, information governance, and regular audits of outputs. The errors and mistakes can be fed back to the artificial intelligence (AI) tool to develop newer, updated, and safer algorithms for effective patient care applications.[7] An interesting observation was that despite less than half of the participants in both groups having any prior experience with ChatGPT, all of them provided their opinions in response to the questionnaire. This makes their responses unscientific, arbitrary, and conjectural, and difficult to be taken seriously. It was unclear from the paper by Parikh et al.,[1] who the healthcare workers were, including doctors (undergraduates/postgraduates/consultants), nurses, paramedics, managers, etc., as well as non-healthcare professionals such as scientists or non-scientists from various professions and backgrounds. This information would be important because each group may have a different understanding and perception of innovative and recently introduced technologies like the AI-driven ChatGPT. It would also be interesting to know whether the questionnaire used in this cross-sectional survey was validated internally (as a pilot) or using validation tools such as Cronbach’s alpha. In addition, we have some comments on question numbers 3, 6, and 9 of the survey: a) “Have you searched online for ChatGPT?” - This question could have been improved by providing a clear definition of what it meant to, “Search online for ChatGPT” to ensure that respondents understood the question. b) “By how much will ChatGPT revolutionize your world in 2023?” - This question may have been difficult for respondents to answer as they may not have had a clear idea of how ChatGPT could potentially impact their world in 2023. Since the chatbot has been introduced very recently, its potential applications are still being explored and are at a very early stage of development. It may have been helpful to rephrase the question to make it more specific or provide examples of potential impact. c) “Stephen Hawking had predicted that uncontrolled development of AI will ultimately lead to the death of civilization and the end of the human race. Do you believe this will be true during your lifetime?” - This question may not be directly related to ChatGPT and could potentially elicit responses that are not relevant to the survey. As ChatGPT is a relatively new technology, it may be premature to ask questions about its impact on the world or to speculate on its potential risks or benefits. We have not seen the full potential of this software as new applications are still evolving. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.

Ähnliche Arbeiten