A Compounding Fingerprint: Telling Abstraction from Exemplar Reuse in Context

Abstract

When a model improves at a structured task after earlier examples in context, has it abstracted the structure or reused the examples? Improvement alone cannot tell: longer contexts, recall and exemplar matching all produce it. We introduce a compounding fingerprint for in-context completion of partial Cayley tables in fresh word labels. Each queried cell is labelled by the weakest axiom class that forces it (Latin, loop or associativity), and histories of earlier complete tables are controlled by a length-matched scramble, held-out isomorphism types, a recall (leak) control and non-associative loops. A preregistered decision rule recovers seven exact idealised learners without error; only held-out transfer cleanly separates abstraction from exemplar matching, a branch of the rule not yet exercised on a learned system. On this format, none of nine checkpoints (ten model–precision configurations) from three families, up to 8B (4–8B in 4-bit), reaches the preregistered Latin-inference threshold in direct-answer mode, and none gains on associativity-forced cells from group histories beyond length-matched non-group ones (t=4 minus scramble: −0.019 to +0.026, every 95% upper bound at most +0.056, against +0.72 for exemplar copying), although Qwen3 at 4–8B recalls seen tables at 0.95–0.96. Small from-scratch transformers learn a count shortcut that a post-hoc count learner matches to within 0.001–0.007 in five of six learned networks. Our positive control, the released In-Context Algebra model, is classified above recall but not Bayes-type: at t=0, Latin- and loop-forced cells score 0.89–0.99, associativity-forced cells 0.53–0.70. Our earlier preprint's compute–retrieve axis, on a format- and length-matched but not vocabulary-matched retrieval/inference contrast, moves significantly in the predicted direction in none of five models and is significantly reversed in three. This project was carried out with substantial AI assistance (Claude, Anthropic); see the Statements section of the paper. Files: the paper (PDF), its LaTeX source, and the preregistration files of the study (compounding-fingerprint-preregistrations.zip). The preregistration files are byte-identical to those in the public repository (SHA-256 sums in the zip and in PREREGISTRATIONS_SHA256SUMS); this deposit timestamps them from its own date and does not backdate them. Code, stimuli, results and preregistrations: https://github.com/Somebodyhere101/compounding-fingerprint