SAMI Omni: Asymmetric Mixture-of-Experts Topology and Memory-Decoupled Multimodal Inference on Commodity Hardware

Abstract

Monolithic scaling of multimodal foundation models has introduced an operational barrier: modern 30B+ parameter architectures typically require distributed enterprise accelerators, restricting local deployment on consumer workstations. While split-device tensor offloading builds upon the existing open-source llama.cpp execution runtime and operates within the physical bounds of the classical Williams Roofline memory bandwidth model, executing high-parameter Mixture-of-Experts (MoE) multimodal models on consumer hardware has remained severely bottlenecked by quantization collapse, routing entropy drift, and prohibitive refusal behaviors. We present a representation engineering and surgical weight adaptation methodology (Representation Engineering & Surgical Model Adaptation) applied to an asymmetric MoE topology. The primary technical contribution of this work is demonstrating that a surgical intervention targeting merely 0.13% of expert feature columns—geometrically stabilized via constrained orthogonal Procrustes alignment and mapped to an asymmetric quantization hierarchy (AsymQ)—renders a massive 30B+ parameter dense-multimodal foundation model fully stable, functional, and de-aligned on an entry-level office workstation with zero gradient retraining on enterprise GPUs.