Transformers have performed strongly in large pre-trained AI systems and many other tasks, but their computational demands have encouraged researchers to seek more efficient designs. This survey examines state-space models, which the authors describe as possible replacements for the self-attention component of Transformers.

The paper explains how state-space models work, reviews existing systems and examines their use in natural-language processing, computer vision, graphs, multimodal and multimedia tasks, point clouds and event streams. It also presents statistical comparisons intended to show how different model structures relate to performance across tasks, and identifies directions for further theoretical and applied research.