CIAFNet: An RGB-D Cross-Modal Interaction and Adaptive Fusion Network for Camellia oleifera Fruit Detection

Abstract

To better address the accuracy bottleneck of RGB-only Camellia oleifera C.Abel fruit detection in complex orchard environments, this paper proposes CIAFNet—a dual-stream RGB-D fusion detection network—and evaluates its potential as a visual front end for relative 3D localization using sensor-measured depth and camera back projection under controlled conditions. With RGB images and depth maps as parallel dual-branch inputs, the network integrates the C3k2_PartialNetBlock for efficient intra-modal feature extraction with reduced computational redundancy, devises the cross-modal interaction and difference-aware adaptive fusion (CIDAF) module for adaptive cross-modal feature fusion, and adopts an SC-EUCB-augmented BiFPN in the neck to optimize multiscale feature aggregation and detail restoration during upsampling. Pseudo-depth maps generated from natural orchard RGB images via Depth Anything V2 were paired with RGB counterparts to build an RGB–pseudo-depth dataset. Synchronized RGB-D data collected by an Intel RealSense D435i under controlled conditions were used to quantify pseudo-to-sensor depth discrepancies and evaluate input adaptability. On the natural orchard test set, CIAFNet achieved 93.33% mAP@0.5 with only 10.49 GFLOPs and 3.80 M parameters. Second-stage fine-tuning improved CIAFNet’s adaptation to D435i-measured depth under controlled conditions. In the subsequent relative displacement consistency experiment, the mean absolute consistency errors along the X, Y, and Z axes were 3.10, 3.15, and 3.27 mm, respectively, and the mean 3D Euclidean consistency error was 5.60 mm. These results demonstrate that CIAFNet improves Camellia oleifera fruit detection using natural orchard RGB–pseudo-depth data and has potential as a visual front end for relative 3D localization under controlled conditions.