The Impact of Prompt Strategies on Specialized Text Translation by Large Language Models: An Empirical Study Using ChatGPT

Abstract

With the rapid advancement of large language models (LLMs), generative AI systems such as ChatGPT have demonstrated substantial capabilities in text generation and translation. Unlike traditional machine translation systems, ChatGPT enables multidimensional control over output through natural language prompts. This study systematically examines the effects of three prompt variables—register, genre, and audience—on the translation quality of specialized texts across three domains: news, medicine, and law. Using BLEU, ChrF++, and COMET as evaluation metrics, we compare translation performance under a baseline prompt (P0) and structured prompts (Pd, Pt, Pa). Results reveal that text-type prompts (Pt) are the most universally effective, yielding the highest performance gains—especially in news texts (BLEU improvement up to 164%). Legal texts show significant sensitivity to register and audience prompts but are inadequately evaluated by BLEU due to near-zero baselines, highlighting the need for semantically aware metrics like COMET. Medical texts benefit from both Pt and Pa, achieving balanced improvements in terminology consistency and semantic adequacy. By constructing a novel framework linking prompt design to output quality, this study offers both theoretical and practical guidance for prompt engineering in domain-specific neural machine translation.

Ähnliche Arbeiten