AI did not pass Finnish plastic surgery written board examination

Abstract

Large language models (LLM) employ neural networks, brain mimicking algorithms, and are trained to understand and generate human language and produce human-like responses.1Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns.. 2023; 11: 887Google Scholar The first, publicly available LLM is ChatGPT (OpenAI, San Francisco, CA, USA), launched Nov 30, 2022. ChatGPT nearly passed the threshold of 60% accuracy in the United States Medical Licensing Exam (USMLE) three exams.2Kung T.H. Cheatham M. Medenilla A. et al.Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.PLoS Digit Health. 2023; 2e0000198Crossref PubMed Google Scholar ChatGPT-3.5 achieved an overall accuracy of 46.8%, while GPT-4 demonstrated an overall accuracy of 76.4% in Korean general surgery board exams.3Oh N. Choi G.S. Lee W.Y. ChatGPT goes to the operating room: evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models.Ann Surg Treat Res. 2023; 104: 269-273Crossref PubMed Scopus (14) Google Scholar However, it must be noted, that the examinations were multiple choice questions. We were curious to test whether two publicly available and free LLMs, specifically ChatGPT and Microsoft Bing would pass the national Finnish plastic surgery written examination. Our written examination is based on three essay questions, supplemented with short-answer questions. Both authors are national board examinators. Previous national board examination questions for all specialties are available freely at (https://www.laaketieteelliset.fi/ammatillinen-jatkokoulutus/erikoislaakari-jaerikoishammaslaakarikuulustelu/ilmoittautuminen/vanhat-kuulustelukysymykset) in Finnish. To successfully pass the examination, a participant must accumulate a minimum of 15 points. Additionally, no question should be assigned a score of 0 points. The responses were scored according to pre-agreed criteria. The scoring was made independently, adhering to the protocol of the real exam. Neither ChatGPT nor Microsoft Bing pass the exam, Table 1. ChatGPT performance was slightly better, Table 2 illustrates the responses. ChatGPT responses were wordy and read well. However, the text seems to be of general knowledge, not deep knowledge. The responses for essay questions ended in a disclaimer.Table 1Questions and performance of ChatGPT and Microsoft Bing in Finnish plastic surgery board examination.QuestionAssessmentMaximum pointsChatGPTBING1a) Quilting Stitches (QS) / Progressive Tension Stitches (PTS): Briefly explain what it means and why to use-0.5 p: skin (scarpa) is sewn to the base (e.g. muscle/ fascia) to reduce dead space.-0.5 p: These can be used e.g. omit drainage.-0.5 p: may reduce seroma formation.-- 0.5 p: used e.g. abdominoplasties and mastectomies.2121b) Venous flow through flap (VFTF)-0.5 p: These flaps consist of skin, subcutaneous tissue, and veenas.-0.5 p: inflow/ outflow from veena. VFTF does not have a native arterial inflow circuit. All flow proceeds to and from the flap through the venous plexus.-0.5 p: are thin and pliable, and their veins are similar in size to the veins of the hand.-0.5 p: raising them results in minimal donor site morbidity2111c) Fillet flap: briefly explain what kind of tab it is and what it can be used for, for example-0.5 p: an amputated part or part that cannot be spared, but can be used to treat a complex trauma or defect caused by a tumor.-0.5 p: axial tabs which can be used as a stem or microvascular graft-0.5 p: multi-tissue grafts-0.5 p: eg. With a crushing injury to the foot, the heel part can be put on the end of the shin stub.20.502) A 75-year-old man who has had a wounding skin lesion at the end of his nose for a long time will be brought to your appointment. A biopsy taken at a health center has revealed squamous cell carcinoma. How do you examine and treat?-1 p: Refinement of the anamnesis: diseases of the patient, condition, ko-operation, self-help-1 p: Is the tumor mobile, what size, is there an indication of a larger growth than the tumor?-1 p: Do you feel something on your neck?-1 p: Are imaging studies needed? MRI/ CT/ neck US?-0.5 p: No routine sentinel lymph node examination-0.5 p: MDT mention, monitoring-0.5 p: What are the treatment options? Skin graft, tabs, cartilage graft-0.5 p: Neck dissection, in case of neck disease62.513) A 34-year-old woman with a referral cutis laxa will come to your outpatient clinic. He has been taking semaglutide (Oxempic) for a year now with the indication for inadequate DM2 control. He has lost 40 kg during this time. She is 166 cm tall and weighs 87 kg when weighed in your practice, so her BMI is 31.6. The main trouble for him is from hanging belly coverings, folds brew and begin to smell. He hasn't had any skin infections that required ab-treatment. You notice a splash of skin below her navel, and a moderate splash of skin above her navel, plus a pubic mound hangs. She is interested in abdominoplasty and has already researched it herself online. Her biggest concern is having children and pregnancy after abdominoplasty. What do you do and what additional An emergency responder calls for a consultation about an elderly 90-year-old woman who has tripped at home and bumped her leg. It has developed a large hematoma. The patient undergoes treatment with Marevan. How do you advise an on-call operator? What issues should be taken into account and how should the lady be treated?-1 p national criteria not met – BMI above 30-1 p weight has not been stable for yet 1 year-1 p Question asked by: do you smoke-1 p No imaging studies-1 p not yet offering abdominoplasty to the patient – national criteria-0.5 p ASA class ≤ 3-0.5 p Pregnancy and childbirth are safe after abdominoplasty60.50.54) An emergency responder calls for a consultation about an elderly 90-year-old woman who has tripped at home and bumped her leg. It has developed a large hematoma. The patient undergoes treatment with Marevan. How do you advise an on-call operator? What issues should be taken into account and how should the lady be treated?-1 p History (when come), clinical picture (size, skin condition/ color)-1 p evacuation in the operating room (puncture is not recommended), possibly left open.-1 p treatment of an open wound, later possibly skin graft.-1 p where to treat (hospital).-0.5 p: pretibial hematoma.-0.5 p: exclusion/consideration of other injuries (e.g. fractures).-0.5 p: INR value.-0.5 p: definition of dermatoporosis: thinning of the skin.61.50.5Total247.55FailFail Open table in a new tab Table 2Answers of ChatGPT and Microsoft Bing in Finnish plastic surgery board examination.QuestionAssesmentChatGPTBING1a)-0.5 p: skin (scarpa) is sewn to the base (e.g. muscle/ fascia) to reduce dead space.-0.5 p: These can be used e.g. omit drainage.-0.5 p: may reduce seroma formation.-0.5 p: used e.g. abdominoplasties and mastectomies.Quilting sutures (QS) or progressive tension sutures (PTS) are surgical techniques used to close a wound after surgery.In the QS technique, special sutures are used, which are placed at the edges of the wound densely and deep into the tissue. These stitches are firmly fixed under the skin, thereby creating an even tension on the wound surface. This will help reduce tension between the edges of the wound and prevent it from rupturing or dissipating.The PTS technique is similar to QS, but involves the gradual or stepless placement of sutures on the edges of the wound. Initially, the sutures are placed near the center of the wound, and then proceed towards the edges. This will help to evenly distribute the tension over the entire wound area and reduce its concentration in one place.QS and PTS techniques help reduce tension between wound edges, thereby preventing complications such as wound dehissiveness (opening) or infection. These techniques are especially used for large surgical wounds, such as the abdomen, chest, or back, where closing the wound can be challenging and tension higher.It is important to note that the use of QS and PTS techniques is considered on a case-by-case basis and may be part of a broader wound care strategy. The surgeon decides on the best method, taking into account the individual needs of the patient and the characteristics of the wound.Quilting sutures (QS) or progressive tension sutures (PTS) are surgical sutures used to reduce seroma formation and other complications in abdominoplasty without the use of drains. PTS stitches are also useful for dead-space management, which minimizes tension in the suture line and reduces seroma formation. However, the use of PTS stitches is variable among plastic surgeons. Springer. (2022). Global Prevalence and Preferences of Progressive Tension Suture Usage in Abdominoplasty: A Systematic Review and Meta-analysis. https://doi.org/10.1007/s00266-022-03146-5 3: PubMed. (2021). Global Prevalence and Preferences of Progressive Tension Suture Usage in Abdominoplasty: A Systematic Review and Meta-analysis. https://pubmed.ncbi.nlm.nih.gov/36316457/Oxford Academic. (2021). Expanding the Progressive Tension Suture Concept: A Study of Its Efficacy in Reducing Seroma Formation in Body Contouring Surgery. https://doi.org/10.1093/asj/sjab1191b)-0.5 p: These flaps consist of skin, subcutaneous tissue, and veenas.-0.5 p: inflow/ outflow from veena. VFTF does not have a native arterial inflow circuit. All flow proceeds to and from the flap through the venous plexus.-0.5 p: are thin and pliable, and their veins are similar in size to the veins of the hand.-0.5 p: raising them results in minimal donor site morbidityVenous flow through flap (VFTF) is a concept that refers to the flow of venous blood through a graft or valve. It is a significant factor that affects the viability and healing of the graft or valve.The purpose of VFTF is to ensure that the tissue to be transplanted or taken as a flap receives sufficient blood flow and oxygen supply to remain viable and heal successfully. Venous blood flows through the blood vessels of the valve or graft and carries oxygen and nutrients to the tissue. At the same time, venous blood removes metabolic by-products and carbon dioxide.VFTF optimization is important in surgeries where tissue repairs or grafts are performed. Surgeons aim to establish adequate blood flow connections between the graft and the receiving area so that venous blood can flow unhindered. This can be achieved, for example, through microsurgical techniques such as anastomoses (connecting blood vessels) or microsurgical suturing.Accurate evaluation and optimization of VFTF are critical to achieving successful tissue repairs or transplants. This helps ensure that the transplanted tissue or valve receives sufficient blood flow to support healing and avoid the risk of transplant failure or complications.Venous flow-through flaps (VFTFs) are useful reconstructive options, especially for the repair of segmental vascular damage. They are relatively easy to collect and give several advantages in the donor area. However, VFTF is based on one central vein, so their survival depends on this vein.PubMed. (2012). The nomenclature of venous flow-through flaps: updated classification and review of the literature. https://pubmed.ncbi.nlm.nih.gov/22434451/1c)-0.5 p: an amputated part or part that cannot be spared, but can be used to treat a complex trauma or defect caused by a tumor.-0.5 p: axial tabs which can be used as a stem or microvascular graft-0.5 p: multi-tissue grafts-0.5 p: eg. With a crushing injury to the foot, the heel part can be put on the end of the shin stub.A fillet tab, also known as a soft tab, is a surgical technique that uses a piece of tissue or graft displaced or wrapped around a damaged or operated area. This will help protect the affected area, promote healing, and improve the esthetic outcome.The fillet tab is most commonly used in the mouth and face area, especially to correct injuries to the lips, mouth or cheeks. For example, if a person has a wound in the lip area, the fillet tab can be taken from nearby soft tissues, such as the oral mucosa or cheek tissue, and transferred to the wound area.The purpose of the tongue is to bring blood circulation and tissue support to the affected area, as well as to help prevent infection and support healing. It also improves the appearance of the wound and reduces the formation of scar tissue.The use of a fillet tab is usually part of complex surgical treatment and requires special skills. The surgeon assesses on a case-by-case basis whether the use of a fillet tab is a suitable option and, based on this, decides on the patient's individual needs and wound characteristics.The fillet tab is a tissue tab containing muscle, fascia, subcutaneous layer, and skin. A tensor fasciae latae (TFL) tab taken from the anterior lateral parts of the thigh is usually used. The length of the tab can be up to 35-40 cm, and the width - 20-25 cm. Taking a tab does not cause significant functional inconvenience.The Fillet tab can be used, for example, in breast prostheses correction and dilated ALT (=anterolateral thigh) tab indications.Duodecim. (2022). Kudoskielekkeet rekonstruktiivisen plastiikkakirurgian arkea. https://www.duodecimlehti.fi/duo96423Duodecim. (2017). Rintaproteesin rikkoutumisepäilyn tutkiminen ultraäänellä. https://www.duodecimlehti.fi/duo13517Duodecim. (2019). ALT-kielekkeen käyttö laajentunut. https://www.duodecimlehti.fi/duo954642)-1 p: Refinement of the anamnesis: diseases of the patient, condition, ko-operation, self-help-1 p: Is the tumor mobile, what size, is there an indication of a larger growth than the tumor?-1 p: Do you feel something on your neck?-1 p: Are imaging studies needed? MRI/ CT/ neck US?-0.5 p: No routine sentinel lymph node examination-0.5 p: MDT mention, monitoring-0.5 p: What are the treatment options? Skin graft, tabs, cartilage graft-0.5 p: Neck dissection, in case of neck diseaseIn the case of suspected squamous cell carcinoma, I would perform the following studies and consider an appropriate method of treatment:Clinical evaluation: I would carefully examine the patient's nasal head ulcer and assess its size, location, and possible symptoms. I would also ask the patient about the presence of other symptoms, such as nasal congestion or nosebleeds.Imaging: I could order imaging studies, such as computed tomography (CT) or magnetic resonance imaging (MRI), to determine the spread of a possible tumor in relation to the nasal cavity or adjacent structures.Histological confirmation: Although you that a biopsy had already been at a health center and squamous cell had been it may be to a new evaluation of the tumor to This will help the and about the characteristics of the depends on the spread of the tumor and the patient's general squamous cell of the nasal area can be in which the tumor is by surgical plastic surgery such as tissue can be if to close the wound and improve the cell may after The patient may be asked to come for to assess wound healing, for of and nasal is important to that this is based on general treatment and the treatment may from case to The best to determine treatment and is based on the patient's based on a and by a cell is the most in the nasal area. Its treatment depends on the spread of the the is and has not spread it can be the has spread to a area, or is examination may use nasal computed tomography (CT) or magnetic resonance imaging to determine the spread of the (2022). (2022). - p national criteria not met – BMI above 30-1 p weight has not been stable for yet 1 year-1 p Question asked by: do you smoke-1 p No imaging studies-1 p not yet offering abdominoplasty to the patient – national criteria-0.5 p ASA class ≤ 3-0.5 p Pregnancy and childbirth are safe after the patient's I would note cutis of the skin in the area to significant weight The patient has symptoms, such as and which his of and clinical evaluation: I will about the patient's weight as well as the of cutis laxa and the of symptoms. I carefully examine the area, assess the of of the skin and the I also the patient's and his in of I ask the patient about and as well as possible complications such as or of the evaluation: I a examination of the patient, in which possible of the or other that may the can be I would consider the patient's general health and the of on I would such as blood and other that I the imaging I would for example, or computed tomography in order to assess the of the coverings, and any other the I with the patient the potential and of especially after I explain that pregnancy can the of the and possibly treatment or of is important to that this is based on general treatment and a and treatment should be made based on the individual needs of the patient and a by the is a repair surgery to and skin from the The will be in pregnancy is but it can the of may be questions and examinations may for the patient have other Is the patient or a the patient have or other the patient have p History (when come), clinical picture (size, skin condition/ color)-1 p evacuation in the operating room (puncture is not recommended), possibly left open.-1 p treatment of an open wound, later possibly skin p where to treat (hospital).-0.5 p: pretibial hematoma.-0.5 p: exclusion/consideration of other injuries (e.g. fractures).-0.5 p: INR value.-0.5 p: definition of dermatoporosis: thinning of the the as the caused by a in a 90-year-old lady and the of the the emergency for about the of the injury and the patient's symptoms. It is important to determine whether the is the injury or if there are other such as a open wound or of is a blood that may the risk of if the patient has any or has had in the while taking Marevan. This will help assess the risk of and the possible of a the size of the and the patient's it is important to that the patient a The can assess the of the for a possible or and, if perform other such as an taking it is important to blood and, if that the on-call with the patient's the of blood thinning and, if the of the and The on-call may be to that the patient treat the with and It is also important to advise the patient to for or other in the area of the or the patient should be for medical is important to that this is based on general treatment and an and treatment should be made based on the patient's individual the of the on-call and the evaluation by the is a blood that the formation of blood For the treatment of you can use and the is large or the patient is treatment with it is important to the INR and, if the of can advise the on-call as patient should be to a patient should be to the and use patient should be to INR and, if the of (2022). (2022). (2022). Marevan. Open table in a new tab Microsoft responses were and it to the end of the of the were and the to in most to Finnish Table the question the to a and to a Finnish not of the not have national as it that Microsoft Bing understand all the questions e.g. venous flow through the We the questions and responses in ChatGPT and that the to the However, of in the most tab flap is at to and is not the plastic use in note the was for the purpose of this responses were not sufficient to pass exam. The responses were or the was or A the relatively performance of in medical board exams.2Kung T.H. Cheatham M. Medenilla A. et al.Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.PLoS Digit Health. 2023; 2e0000198Crossref PubMed Google N. Choi G.S. Lee W.Y. ChatGPT goes to the operating room: evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models.Ann Surg Treat Res. 2023; 104: 269-273Crossref PubMed Scopus (14) Google Scholar The questions were and to and that be one of the performance of in this perform according to or or the of LLMs, and based on the may be or not so flow through to be for LLM to for the human the order is on the known as flow through and not flow This all to human The was in questions and that required as a plastic as ChatGPT ended with Table It is to in the responses also in the as and are for medical What if your patient from Google to A of the and of responses to health 2023; Scholar The that is in use in plastic surgery in from the from which the its Microsoft Bing or where the was was to consider national and most the or were but why was used as a This has been as of using not and such as PubMed. et significant between human and M. ChatGPT or with over accuracy using 2023; Scholar text was complex to human to produce responses to questions. if it is used for The that is in the of medical is not with can be a for of may not be to what is correct and what is in the responses and how to use the in real may generate questions and With the may the of the question or it

Ähnliche Arbeiten