Hyams Foundation

The Missing Medium: A Transactional Residual Stream for Transformer Training

Modern training treats the transformer residual stream as an anonymous accumulator: any kernel may execute h += \delta, and no record of the write survives the step. Training observes the consequences of internal writes — loss curves, evaluations, activation probes — never the wr...