Oliver Zahn

Sapience: A Hybrid Architecture for Long-Context Reasoning and Continual Learning

Frontier models approach long-context reasoning by scaling the transformer context window: one of the several memory systems the brain evolved, scaled alone. We present Sapience, a hybrid architecture that builds the missing ones: a substitutable transformer cortex coupled to hip...