Curtis Sandoval

Stratum: Lie Field Layers for Grid Logic Puzzles in Small Transformers

We introduce Lie field layers, which add a decaying, optionally rotating recurrence over sequence positions to a transformer’s residual stream, and evaluate them on abstract-vocabulary grid logic puzzles. This version 1.1 corrects errors in version 1.0; every change is listed in...