SFD-KD: Structured Feature Decoupling Knowledge Distillation for Fracture Detection

Abstract

Accurate fracture detection in medical imaging is pivotal for intelligent orthopedic diagnostic systems, yet deploying high-capacity detection models on resource-constrained platforms remains a critical cyber-physical challenge. Conventional feature distillation based on Mean Squared Error (MSE) performs point-wise feature regression but does not explicitly model higher-order statistical relationships, which may lead to over-smoothed responses and weaken the preservation of subtle fracture structures. To overcome this systemic limitation, we propose Structured Feature Decoupling Knowledge Distillation (SFD-KD). This framework decomposes teacher features into linearly combined first-order semantics and multi-order statistics, bypassing computationally expensive covariance modeling. Concretely, SFD-KD integrates three specialized modules: the Multi-Order Statistic Extractor (MOSE) for multi-order structural alignment, First-Order Statistic Extractor (FOSE)-SVD for teacher-side semantic extraction via Singular Value Decomposition (SVD), and FOSE-FFT for student-side semantic stabilization using Fast Fourier Transform (FFT) with a Learnable Spectral Filter (LSF). Extensive experiments on the GRAZPEDWRI-DX and FracAtlas datasets demonstrate that SFD-KD consistently outperforms vanilla KD by +1.4~+4.4% mAP@0.5:0.95 across both Faster R-CNN and YOLOv8 frameworks. Ablation studies confirm the efficacy of the proposed statistical decoupling, while visualizations reveal superior preservation of multi-order fracture patterns. Notably, SFD-KD enables robust deployment in clinical cyber-physical systems with negligible inference overhead. Our code is available at: https://github.com/6720230811/SFD-KD.