Geometric Deformation Does Not Imply Task-Relevant Information Degradation in Neural Representations

Abstract

Geometric change in a neural representation is often treated as evidence that information has been altered or lost, although these are distinct properties of representation dynamics. We formalize this distinction by separating geometric deformation, Delta(t), from task-relevant information degradation, delta(t), along a recursive trajectory of observable representations. We operationalize geometric deformation using adjacent-layer centered kernel alignment (CKA), task-relevant accessibility using layer-wise linear probes, and representational dimensionality using entropy-based effective rank. In a proof-of-concept study of BERT-base on SST-2 over five random training subsamples, mean linear-probe accuracy rises from 0.733±0.004 at layer 1 to 0.831±0.016 at layer 11 before decreasing to 0.816±0.006 at layer 12, while effective rank continues to increase from 235.0 to 437.8. Adjacent-layer geometric deformation follows a different trajectory and does not systematically coincide with probe degradation. These results provide empirical evidence that geometric reorganization, effective dimensionality, and task-relevant linear accessibility can follow distinct trajectories. Representation change should therefore not be interpreted as information degradation without an explicit task-relevant preservation measure.