NB-Net: A Biologically-Inspired Framework for Neural Network Width Expansion

Abstract

Abstract Deep learning has achieved remarkable success across various fields, however, most research in this area has primarily focused on increasing network depth, leading to a relative lack of systematic exploration of network width. Inspired by the structure of biological neural systems, widening a neural network can enhance the resolution of the feature space, enabling the capture of richer and more fine-grained visual features. To this end, we propose a novel architecture called Neuron Bundle Network (NB-Net), featuring a groundbreaking dual 1×1 convolution fusion mechanism that revolutionizes multi-branch feature integration. NB-Net is inspired by the morphological structure of dendrites and axons in biological neurons. It aggregates multiple parallel processing paths using diverse convolutional layers and attention modules through our innovative dual 1×1 convolution design. Our dual 1×1 convolution fusion mechanism serves as the core innovation, providing superior feature integration compared to traditional single-stage merge operations by enabling gradual dimensionality reduction and enhanced gradient stability. The framework introduces several key innovations, including the dual 1×1 convolution two-stage merge mechanism for stable gradient flow, adaptive learning rate scaling for attention integration, and a systematic approach to kernel diversity orchestrated through our fusion strategy. Ablation studies demonstrate that the dual 1×1 convolution mechanism drives performance gains, enabling competitive results with efficient structured sparsity and feature fusion.