AFF-net: adaptive feature fusion network for efficient medical image segmentation

Abstract

Medical image segmentation is essential for clinical diagnosis and treatment. Existing convolution and self-attention-based segmentation methods suffer from heavy computational overhead, while complex lesion morphology and blurred anatomical boundaries further degrade segmentation performance. To address the accuracy-efficiency trade-off in medical segmentation, this work proposes a lightweight adaptive feature fusion network (AFF-Net) with an optimized adapter fine-tuning paradigm. AFF-Net embeds a plug-and-play multi-scale perception adapter (MSPA) in the backbone to achieve efficient multi-scale feature learning and avoid feature drift caused by conventional full fine-tuning on limited medical data. Furthermore, we integrate a lightweight multi-scale convolutional attention block (LMCAB) into the efficient multi-scale convolutional attention decoder (EMCAD) to adaptively fuse multi-level spatial and channel features and mitigate information attenuation. Benefiting from this streamlined embedded decoder design, the network strengthens semantic representation and effectively refines ambiguous anatomical edge details. Evaluated on Synapse and ACDC datasets, AFF-Net achieves competitive or superior performance against state-of-the-art methods with lower computational costs. Ablation experiments validate the efficacy of core modules and the adapter fine-tuning strategy, demonstrating the practical clinical application potential of the proposed lightweight network. The codes and models are available at https://github.com/wangru1026/AFF-Net .