AI & Computingarticle2026-10-21

State space model for new generation network alternative to transformers: a survey

Open access28 citations

Abstract

Abstract In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance in pre-trained large models and various downstream tasks. However, the enormous computational demands of Transformers have deterred many researchers. To reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the state space model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. This paper provides the first comprehensive review of SSM works and offers experimental comparisons and analyses to better demonstrate SSM’s features and advantages. First, we provide a detailed description of the principles to help the readers quickly grasp the key ideas of the SSM. Then, we will review existing SSMs and their various applications, including those in the fields of natural language processing, computer vision, graphs, multi-modal and multi-media, point cloud and event stream, and other domains. Next, we provide statistical comparisons and analyses of these models to help readers understand how different structures affect performance on various tasks. Finally, we suggest potential research directions that could further advance the development of the theoretical model and propose possible research points in this direction to better promote the application of SSMs.

// Source

View paper (DOI)Open access versionOpenAlexVisual IntelligencePublished 2026-10-21

Authors: Xiao Wang, Shiao Wang, Yuehang Li, Wentao Wu, Rong Yao, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, Ziwen Wang, Chenglong Li, Yaowei Wang, Yonghong Tian, Jin Tang

Institutions: Peking University, Harbin Institute of Technology, Anhui University, Peng Cheng Laboratory