Large Language Models for Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving

Every’s Master Plan


View a PDF of the paper titled Large Language Models for Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving, by Andong Chen and 7 other authors

View PDF
HTML (experimental)

Abstract:Different from the traditional translation tasks, classical Chinese poetry translation requires both adequacy and fluency in translating culturally and historically significant content and linguistic poetic elegance. Large language models (LLMs) with impressive multilingual capabilities may bring a ray of hope to achieve this extreme translation demand. This paper first introduces a suitable benchmark (PoetMT) where each Chinese poetry has a recognized elegant translation. Meanwhile, we propose a new metric based on GPT-4 to evaluate the extent to which current LLMs can meet these demands. Our empirical evaluation reveals that the existing LLMs fall short in the challenging task. Hence, we propose a Retrieval-Augmented Machine Translation (RAT) method which incorporates knowledge related to classical poetry for advancing the translation of Chinese Poetry in LLMs. Experimental results show that RAT consistently outperforms all comparison methods regarding wildly used BLEU, COMET, BLEURT, our proposed metric, and human evaluation.

Submission history

From: Andong Chen [view email]
[v1]
Mon, 19 Aug 2024 12:34:31 UTC (9,676 KB)
[v2]
Wed, 16 Oct 2024 12:15:39 UTC (9,552 KB)
[v3]
Thu, 17 Oct 2024 02:17:57 UTC (9,552 KB)
[v4]
Mon, 30 Dec 2024 07:26:14 UTC (19,291 KB)



Source link
lol

By stp2y

Leave a Reply

Your email address will not be published. Required fields are marked *

No widgets found. Go to Widget page and add the widget in Offcanvas Sidebar Widget Area.