An Optimized Audio–Visual Deepfake Detection Framework Using Mamba Vision Transformers with Linear Complexity

Abstract

The rise of advanced deepfake technologies presents new challenges to media authenticity, making it harder to spot manipulated videos through traditional approaches. Current approaches have a quadratic time complexity O(n2), poor cross-dataset generalization with 20 to 35% accuracy degradation, and inadequate audio–visual fusion mechanisms. This paper presents Mamba-AVGuard, a novel audio–visual deepfake detection framework leveraging Mamba Vision Transformers with selective state-space modeling to achieve linear O(n) complexity within the Mamba encoder streams. The novel framework combines a 12-layer visual Mamba encoder for spatial and temporal facial inconsistency analysis with an 8-layer lightweight audio Mamba encoder for prosodic inconsistency analysis, fused via a cross-modal attention block with ±33 ms temporal synchronization. Experiments conducted on two large-scale audio–visual deepfake benchmarks, FakeAVCeleb and AV-Deepfake1M, demonstrate that Mamba-AVGuard outperforms compared methods with 99.92% in-dataset accuracy and 98.47% cross-dataset accuracy on these two benchmarks. The system achieves a processing speed of less than 1.8 s for 30-s videos, with 56% less memory than the best competing transformer-based method, facilitating a throughput of more than 1200 videos per hour for social media verification and forensic workstations.