Gianluca Gernone

The Hidden Geometry of Transformer Weights: A Canonical Basis for Interpreting Transformer Language Models

Version 2 of the paper "The Hidden Geometry of Transformer Weights". Rotating a Transformer language model into a coordinate system alignedwith its own weight matrices — the canonical basis — reveals a hiddeninternal structure. Version 1 established the descriptive phenomena.Vers...