SAMI Chimera: Sub-Neuronal Weight Surgery, Chimeric Base-to-Chat Synthesis, and Tensor Isometry Certification in DeepSeek-MoE-16B

Abstract

Post-training preference alignment (RLHF, DPO, KTO) enforces refusal behaviors by establishing persistent attractors in intermediate transformer representations. Current refusal removal techniques, notably directional ablation, operate macroscopically by subtracting a 1-dimensional refusal vector from entire layer weight matrices, modifying 100% of targeted dense parameters and inducing severe reasoning degradation (-10.7% on GSM8K). Building upon the representation engineering framework established in SAMI Omni, this technical report presents Sub-Neuronal Mechanistic Weight Editing (SN-MWE) coupled with Constrained Orthogonal Procrustes Manifold Recovery, evaluated across the DeepSeek-MoE-16B architecture (1,782 experts and 5,466 tensors). By decomposing SwiGLU Feed-Forward Networks into discrete intermediate channel triplets, we prove that refusal mechanisms reside in an ultra-sparse subset (<0.14%) of intermediate channels. Specifically, we formulate and empirically validate:1. Sub-Neuronal Refusal Isolation and Excision: Neutralizing exactly 113 refusal channels across 16 critical experts, modifying only 0.13% of parameters while leaving over 99.86% of weights bit-for-bit untouched.2. Foundation Base Channel Transplantation (Chimeric Synthesis): Refusal-steering channels in the aligned Chat checkpoint are surgically replaced with unaligned Foundation Base columns, preserving conversational syntax while maintaining 100% router gating invariance.3. Constrained Orthogonal Procrustes Stabilization & Isometry Certification: Representations are geometrically stabilized via proper orthogonal rotations in SO(d) with det(R) = +1. A full-model audit across all 5,466 tensors certifies zero numerical anomalies (NaN/Inf) and a mean norm isometry ratio of 0.9991.4. Asymmetric Mixed-Precision Quantization (AsymQ): Packaging dense trunks and router gates at Q8_0 while compressing routed experts to Q3_K/Q4_K, achieving a 9.0–9.3 GB production footprint with 0.0% refusal rate and near-zero reasoning degradation (81.9% on GSM8K, Δ = -0.2%), executed statically in under 4 GB RAM/NVMe.