Coupled Fourier Neural Operator and Vision Transformer Bottleneck for Parameter-Efficient Brain Tumor Segmentation

Abstract

Segmenting brain tumor subregions in multimodal MRI is difficult due to severe class imbalance and scarce annotated data, and current state-of-the-art models require 21.3–31 million parameters to reach a whole-tumor Dice of 0.906–0.921. We propose a parameter-efficient 2D encoder–decoder coupling a Fourier Neural Operator (FNO) and a Vision Transformer (ViT), trained with BraTSPipeline, which raises throughput from 257 to 16,040 slices per epoch, and two composite loss functions penalizing false negatives 2.3× more than false positives. On BraTS 2020, the 2D variant (5.6 M parameters) achieves whole-tumor (WT), tumor-core (TC), and enhancing-tumor (ET) Dice of 0.8985, 0.8263, and 0.7568 on the 56-case held-out test set, using 3.8–5.6× fewer parameters than published architectures. Encoding three consecutive axial slices as 12 channels (2.5D, 20.5 M parameters) raises WT to 0.9117, within 0.010 of the best published 2D result on BraTS 2020, Mod-R2AU-Net (WT = 0.921); a 4.6 M-parameter ablation without the ViT reaches WT of 0.9106 and TC of 0.8428, while the ViT adds 0.043 ET Dice.