Predictive Structure is Distributed Across Depth in EEG Foundation Models

Abstract

This work investigates a common but insufficiently understood phenomenon in EEG foundation models: intermediate representations often outperform the final layer when pretrained encoders are used as frozen feature extractors. The study disentangles three possible sources of this effect—readout mismatch at the final layer, complementary predictive structure preserved at intermediate depths, and the cross-dataset reusability of depth-dependent representations. Across five EEG foundation models and multiple downstream tasks, improved readout strategies recover substantial final-layer performance, yet intermediate layers retain additional predictive utility in most settings. Residual analyses further show that intermediate representations contain task-relevant variation that cannot be fully recovered from the final layer and can improve prediction when combined with final-layer features. Cross-dataset experiments additionally demonstrate that source-selected layers and compact task-relevant directions derived from intermediate residuals remain useful after transfer. Overall, the results suggest that predictive information in EEG foundation models is distributed across depth rather than concentrated in a single canonical representation, and that its utility depends jointly on layer depth, readout design, and cross-layer feature integration.