D56. Large Language Models for Perceptual Speech Clinical Data Extraction in 50 Cleft Palate Speech Language Pathology Notes

Abstract

PURPOSE: Speech-language pathology assessments are vital in supporting plastic surgeons in monitoring pediatric patients with cleft palate and velopharyngeal insufficiency. Recent advancements in large language models (LLM) may potentially expedite the time required for manual extraction of a patient’s functional speech status. This study evaluates the efficacy of an in-house developed GPT-4 model (VERSA) in extracting concepts from speech notes. METHODS: Speech-language pathology assessments are vital in supporting plastic surgeons in monitoring pediatric patients with cleft palate and velopharyngeal insufficiency. Recent advancements in large language models (LLM) may potentially expedite the time required for manual extraction of a patient’s functional speech status. This study evaluates the efficacy of an in-house developed GPT-4 model (VERSA) in extracting concepts from speech notes. RESULTS: Our model had a 94% accuracy rate in extracting patient’s speech diagnosis, surgical history, SLP assessments, and noting the absence of pertinent information for evaluation from deidentified speech notes. The LLM inaccurately identified speech acceptability and resonance ratings for two notes and was unable to extract surgical history in one note. CONCLUSION: Findings highlight the potential of LLMs in enhancing the efficiency and reliability of SLP clinical assessments by streamlining the extraction of diagnostic and evaluative information from pediatric speech pathology notes. This model reduces the time spent manually retrieving information and has potential to aggregate data to see trends, and comprehensively extract data for research. Future growth of this LLM will focus on refinement to improve accuracy and applicability.

Ähnliche Arbeiten