Ziheng Zhou

Zero-Interference Associative Recall in a Single Linear Recurrent Language Model: Frozen Stable Keys, a Trigger Gate, and Decoupled Training

Autoregressive language modeling must do two things at once: compress a mostly-local context into a state that drives fluent next-token prediction, and, on occasion, re-encounter a previously seen key and emit the exact value bound to it. Linear-time recurrent and state-space mod...