Mechanistic Tomography: Designed Measurement for Control-Oriented Interpretability

Abstract

Mechanistic interpretability seeks quantities that a model does not expose directly: represented states, component effects, interactions, and responses to interventions. Patching, gradients, Hessian-vector products, and subset interventions obtain different measurements under different access assumptions and may target different quantities. We formulate their shared measurement problem as mechanistic tomography: the design and analysis of measurements for recovering internal mechanisms and intervention effects. For a chosen basis and intervention family, each measurement records an intervention, an unknown effect map, and the response that the chosen linear model misses. This gives methods with different targets a common set of questions and a practical procedure: begin with the least costly available measurements, test predictions on held-out interventions at the intended scale, calibrate simple finite-scale mismatch, and expand the measurement family when structured residuals remain. The paper develops results on measurement error, perturbation design, calibration dimension, sparse aggregate recovery, and interaction-aware measurement. It validates the framework in controlled belief-state and planted-effect models, Tracr programs, and GPT-2-small IOI. Control serves as the principal validation setting because an estimate that guides an intervention acts as an observer, making estimation error consequential. Preprint. An arXiv version is forthcoming.