Assessing DeepSeek R1 and ChatGPT 4.5 in Arabic-English literary translation: performance, challenges, and implications

Abstract

Artificial Intelligence (AI) tools such as DeepSeek R1 and ChatGPT 4.5 have emerged as promising aids in Arabic-English literary translation. This study aims to compare the translation performance of these two systems using a mixed-methods approach. Quantitative analysis was conducted through five established evaluation metrics, BLEU, COMET, METEOR, BERTScore, and sacreBLEU, to assess accuracy, fluency, and semantic coherence. Complementing these measures, a qualitative evaluation was carried out with 80 undergraduate students from Cadi Ayyad University, who critically assessed anonymized AI-generated translations against their versions and published human translations using a structured rubric. Results indicate that DeepSeek R1 achieved consistently higher automated metric scores across literary genres (novels, plays, poems). However, qualitative analysis highlighted persistent challenges in pragmatic coherence, cohesion, and emotional depth, especially in poetry and dramatic texts. While DeepSeek R1 demonstrated potential in lexical accuracy and fluency, significant human intervention remains essential for achieving high-quality literary translations. Future research should integrate larger-scale human evaluations to comprehensively assess the capabilities and limitations of AI translation tools in diverse literary contexts.

Ähnliche Arbeiten