Deep learning driven English neural machine translation system optimization

Abstract

Existing deep learning (DL)-driven English neural machine translation systems treat translation quality and inference efficiency as independent optimization objectives that resist synergistic improvement. This paper proposes a joint optimization framework based on the Multiple Gradient Descent Algorithm (MGDA) that trains a Transformer using translation quality loss and an efficiency loss modeled by a delay estimation function. The Transformer’s self-attention mechanism models cross-distance syntactic relationships globally, and the proposed framework imposes no language-specific inductive bias on the underlying architecture. Moreover, the delay estimation function continuously and differentiably models the structural property of the attention weight matrix, thereby introducing a regularization objective that participates in gradient backpropagation. In each training procedure, MGDA solves a convex quadratic programming problem to synthesize a joint update direction from two gradient vectors of the loss function. This paper intrinsically balances the optimization of translation quality and inference efficiency in a Pareto-optimal sense and guides the model toward low-latency structural characteristics after iterative training. Experimental results show that, in the trade-off between translation quality and inference efficiency, the proposed MGDA model achieves a BLEU score of 28.67, while reducing inference latency to 93.51 milliseconds per sentence. This framework offers a multi-objective optimization solution for DL-driven NMT systems, enabling sustained accuracy and efficiency across morphologically diverse language pairs.