PMR-Q&A: Development of a Bilingual Expert-Evaluated Question–Answer Dataset for Large Language Models in Physical Medicine and Rehabilitation

Abstract

<b>Objectives</b>: This study presents the development of a bilingual, expert-evaluated question-answer (Q&A) dataset, named PMR-Q&A, designed for training large language models (LLMs) in the field of Physical Medicine and Rehabilitation (PMR). <b>Methods</b>: The dataset was created through a systematic and semi-automated framework that converts unstructured scientific texts into structured Q&A pairs. Source materials included eight core reference books, 2310 academic publications, and 323 theses covering 15 disease categories commonly encountered in PMR clinical practice. Texts were digitized using layout-aware optical character recognition (OCR), semantically segmented, and distilled through a two-pass LLM strategy employing GPT-4.1 and GPT-4.1-mini models. <b>Results</b>: The resulting dataset consists of 143,712 bilingual Q&A pairs, each annotated with metadata including disease category, reference source, and keywords. A representative subset of 3000 Q&A pairs was extracted for expert validation to evaluate the dataset's reliability and representativeness. Statistical analyses showed that the validation sample accurately reflected the thematic and linguistic structure of the full dataset, with an average score of 1.90. <b>Conclusions</b>: The PMR-Q&A dataset is a structured and expert-evaluated resource for developing and fine-tuning domain-specific large language models, supporting research and educational efforts in the field of physical medicine and rehabilitation.

Ähnliche Arbeiten