Can AI translate culture? An empirical assessment of LLMs’ translation of Chinese legal culture-specific items

Abstract

It has long been claimed that machines can not understand and translate culture, but the fast evolving of AI technology, particularly the Large Language Models (LLMs), presents new possibilities. This empirical study investigates and compares the performance of two state-of-the-art LLMs, namely, DeepSeek-V3 and ChatGPT-4o, in rendering into English a group of traditional Chinese legal culture-specific items (LCSIs), and examines to what extent human-guided revision prompting can improve their quality of translation. Fifty authentic Chinese sentences containing fifty-six traditional LCSIs were selected as the source texts and translated by the two models, whose output was assessed manually based on a translation quality assessment (TQA) model featuring accuracy and understandability. A multi-stage human-guided revision process was adopted to elicit the models’ best possible performance, and Wilcoxon Signed-Rank Tests were used to compare the data. The results show that there was no significant difference in the translation quality of the two models, though DeepSeek performed slightly better in the initial stage. Human-guided revision prompting significantly improved the performance of both models, highlighting the importance of human intervention and guidance. These findings indicate that, although current LLMs show strong potential in translating culturally embedded legal content, high-quality translation in this specialized domain still relies heavily on human expertise.