Distributed Mechanistic Interpretability at Scale: Activation Streaming, Split-Layer Inference, and Distributed Sparse Autoencoder Training

Abstract

ActStream: a two-node system that splits inference and analysis, streams intermediate activations over a binary wire protocol, and trains sparse autoencoders without staging activations to disk. Thunderbolt measured at 2.150 GB/s, 86% of negotiated rate and 26x the LAN path, with p99 latency within 1.5x of median. Includes a feature-based safety classifier evaluated against an LLM-judge baseline on PKU-SafeRLHF.