Allan Ochola

Distributed, Not Sparse: Decoder-Basis Geometry of Toxin-Related Directions in a Protein Language Model

Sparse autoencoders (SAEs) are increasingly used to identify interpretable features in protein language models, but a failure to recover stable sparse features can have at least two explanations: the relevant biological information may be absent from the underlying representation...