CRNN-V2: speech emotion recognition from audio signals using deep learning techniques

Abstract

In human–human communication, emotions are naturally perceived through facial expressions, gestures, and speech patterns. However, enabling machines to accurately recognize human emotions during human–machine interaction remains a significant challenge. Speech Emotion Recognition (SER) has therefore emerged as an important research domain, aiming to identify emotional states directly from vocal signals and speech intonation. The proposed framework combines Convolutional Neural Networks (CNNs) for local feature extraction, Bidirectional Gated Recurrent Units (BiGRUs) for temporal sequence modeling, and a multi-head attention mechanism to capture both spectral and contextual information from speech. Differential Privacy is incorporated during training through gradient clipping and controlled noise injection to protect sensitive speech data while maintaining high recognition performance. The preprocessing pipeline includes noise reduction, signal normalization, extraction of 40-dimensional Mel Frequency Cepstral Coefficients (MFCCs), generation of \(128 \times 128\) log-Mel spectrograms, and feature scaling to improve acoustic representation. The model is evaluated on the Speech Emotion Recognition (SER) benchmark datasets, which contain eight emotion classes: Angry, Calm, Disgust, Fearful, Happy, Neutral, Sad, and Surprised. Performance is assessed using Accuracy, Precision, Recall, F1-score, Word Error Rate (WER), Privacy Score, privacy budget ( \(\epsilon\) ), and privacy leakage analysis. Experimental results demonstrate that the proposed Hybrid CRNN-V2 achieves 98.56% accuracy, 99.56% precision, 100% recall, 99.11% F1-score, 3.56% WER, and a Privacy Score of 98.92% while maintaining strong privacy guarantees. Comprehensive comparative experiments show that the proposed model consistently outperforms conventional machine learning algorithms, including SVM and XGBoost, deep learning architectures such as CNN, RNN, LSTM, GRU, BiLSTM, BiGRU, CNN+LSTM, CNN+GRU, CNN+Attention, Seq2Seq, and YOLO-Audio, as well as recent speech foundation models including OpenAI Whisper, Whisper Large-v3, Qwen-ASR, Moonshine, ElevenLabs Scribe, Deepgram Nova-3, AssemblyAI Universal, and Gemini 1.5 Pro. The proposed framework provides a secure, efficient, and highly accurate SER solution for privacy-sensitive applications, including healthcare, intelligent virtual assistants, affective computing, customer service analytics, smart surveillance, and intelligent human-machine interaction.