Shared Safety-Boundary Representations and Recovery after Single-Site Ablation in a Hybrid Attention Model
Abstract
Can shared activation structure elicited by safety-related inputs be interpreted as a removable refusal-control state? We report a single-model study combining paired representations, independent factor manipulations, a frozen behavioral experiment, and downstream trajectory audits. The locally quantized model has a measured qwen35 architecture with a 64-block trunk, hidden dimension 5120, 48 Gated DeltaNet blocks, and 16 full-attention blocks. Across 34 answerable/inappropriate-execution pairs, generation-start differences exhibit cross-theme consistency: calibration-to-test mean theme cosines are 0.841 and 0.860 at L36 and L40, with positive projections for all 16 test pairs and two contexts from one held-out theme. Independent batches support adverse-purpose and procedural-means sensitivity, but their directions overlap substantially. A three-dimensional subspace, S3, explains 59.6% and 63.5% of pooled test-difference energy, compared with 47.8% and 52.4% for procedure-matched sign-randomized nulls. A frozen behavioral experiment on 18 new context pairs and 36 inputs completes 864 runs; neither 324 target-ablation cells nor 324 equal-L2 random-control cells change fulfillment grades. Exploratory trajectories subsequently show near-complete removal at the intervention site followed by reappearance, reaching approximately 0.79-0.84 baseline alignment at the input to block 63. Recovery and behavioral stability co-occur, but recovery has not been established as necessary for refusal. These results separate interpretable, nonorthogonal input-side structure from deletion persistence and behavioral controllability under a specific intervention. Preprint v1, not peer reviewed. Public materials include the manuscript, summary figures, and evidence manifest, not the original reports or raw arrays. No institutional affiliation is listed.
// Source
Authors: Jun Cai