Cross-Model Stitching Transfers Answers, Not Computation

Abstract

Activation stitching describes the technique of grafting one model’s hidden state into another’s through a learned map. Its reasoning is grounded in universality, under which shared circuitry between models is assumed to enable seamless transfer. Prior work validates this technique, showing that features, probes, and steering vectors can survive transfers. However, what a stitch actually transfers, the question with likely the largest implication for universality, is yet to be explored in depth. Our paper aims to establish this boundary with a task-supervised linear stitch at a single layer, grafting a large donor’s residual state into a smaller, same-family recipient across three model pairs (Gemma-2 9B→2B, Qwen2.5 7B→0.5B, Llama-3 8B→1B) and four task families (arithmetic, symbolic binding, state tracking, and GSM8K). Our experiments find that the stitch is a functional channel. It confers the donor’s answer on up to 88.6% of otherwise-unsolvable problems in our primary arithmetic evaluation. Additionally, a breadth of controls shows that what crosses is not capability, but rather a precomputed answer that the donor passes onto the recipient. The stitch transfers the donor’s precomputed answer to the recipient at the graft site, with conferral collapsing when the answer subspace is ablated out. We also find, however, that this presence alone is not sufficient for transfer. The stitch must encode the answer in a way that is interpretable to the recipient, which is inherently lossy. Finally, we also find that multi-step or multi-digit computation is not computed in prefill, and therefore fails to transfer until prior answer tokens are already computed.