Elaheh Sabbaghi

Post-Stability Representational Change]{When Accuracy Stabilizes, Representations Still Move: Learning-Rule-Dependent Geometry of Post-Stability Representational Change

Neural networks are usually considered "converged" once validation accuracy plateaus. This preprint asks whether the internal representations have actually stopped changing too and finds that, often, they haven't. We train shallow MLPs on binary MNIST using two learning rules: st...

Post-Stability Representational Change]{When Accuracy Stabilizes, Representations Still Move: Learning-Rule-Dependent Geometry of Post-Stability Representational Change

Neural networks are usually considered "converged" once validation accuracy plateaus. This preprint asks whether the internal representations have actually stopped changing too and finds that, often, they haven't. We train shallow MLPs on binary MNIST using two learning rules: st...