Efficient Mechanistic Circuit Analysis of Open-Weight LLMs on Apple Silicon

Abstract

Replication and extension of the GPT-2 Small IOI circuit analysis on Llama-3.2-3B and Pythia-1.4B using an MLX activation-patching harness on a single Apple Silicon machine. Circuits are faithful on the standard single-template dataset (F=0.663 and 0.854) but collapse on a 15-template dataset (0.228 and -0.028) while both models continue to perform the task above 90% of their original logit difference. Single-template IOI circuits are properties of a sentence frame rather than of the task.