Cross-lingual phoneme-aware audio synthesis framework for low-resource Yoruba word generation

Abstract

The scarcity of annotated data and unique linguistic challenges have made low-resource African languages largely bypassed in the development of high-fidelity TTS systems. This paper presents a novel cross-lingual, phoneme-aware audio synthesis framework for the Yorùbá language, a tonal and morphologically rich West African language with limited computational resources. Our approach leverages a transfer learning paradigm where an English-trained FastSpeech 2 model is adapted using a custom Yorùbá phoneme-to-tone transcription pipeline, combined with tonal embeddings and speaker conditioning. A high-quality but limited dataset consisting of 6 h of annotated Yorùbá speech from 12 native speakers was used to fine-tune the system. Extensive evaluations, including mel cepstral distortion, tone classification accuracy, and Mean Opinion Score (MOS) native speaker evaluations demonstrate that the system can synthesize intelligible and tonally accurate Yorùbá speech despite limited data. Attention-based alignment visualizations and SHAP interpretability analyses also confirm that the model captures the phoneme-tone dependencies which are critical for the language intelligibility. This work makes an important contribution to the field of low-resource speech synthesis, with wide implications for linguistic preservation, accessibility, and inclusive AI for underrepresented languages.