Hideki Ishiyama

Algebraic Geometric Empirical Process (AGEP) Theory of Deep Learning Dynamics

Modern Transformer architectures have achieved outstanding empirical success, but their parameter spaces are fundamentally characterized by complex algebraic singularities, such as rank collapse in self-attention mechanisms, which invalidate classical regular statistical asymptot...