Kimi K3 Under Compute Constraints: Implementing MergeMoE to Take 14 Shards to 5GB on Consumer Hardware

Abstract

This paper documents the end-to-end effort of reducing Kimi K3, a 2.8-trillion-parameter, 896-expert Mixture-of-Experts model, to a single serving-grade ~5 GB, 8-layer slice that runs on a 2019 Lenovo ThinkPad L390 laptop (Intel Core i5-8265U, 7.4 GB usable single-channel DDR4, no discrete GPU). The project proceeds in three acts: first, Unsloth's 594 GB, 14-shard 1-bit GGUF release is merged with a purpose-built expert-merging utility into a single dense 66.3 GB model (62.23 B parameters, two super-experts per layer); second, that model is pruned and quantized into an 8-layer serving slice through a path documented as a study in failure modes, including a silently corrupted quantization intermediate, a degenerate rebuild, a layer-pruning tool bug, and a metadata array mismatch; third, the working slice is measured exhaustively: ~2.5-2.6 tokens/s single-stream decode, ~5.3 tokens/s aggregate across six batched sequences, a ~458K tokens/day sustained ceiling, and a definitive single-channel-memory diagnosis from measured memcpy bandwidth. The central findings are that a frontier-scale MoE can be brought down to a running 5 GB artifact on decade-old consumer silicon, that the machine's memory subsystem (not the model or the kernels) is the binding constraint, and that the careful post-mortem of the corrupted-slice incident is the most transferable engineering output. The serving slice was not end-to-end benchmarked for quality; benchmarking against the base K3 and the K3 1-bit release is required, and RL/fine-tuning the slice is planned but not implemented. Author: Haitham al-Waisy Contact: haithamalwaisy@gmail.com | X/Twitter: https://x.com/HaithamalWaisy | YouTube: https://youtube.com/@haithamal-waisy