Haitham al-Waisy

Kimi K3 Under Compute Constraints: Implementing MergeMoE to Take 14 Shards to 5GB on Consumer Hardware

This paper documents the end-to-end effort of reducing Kimi K3, a 2.8-trillion-parameter, 896-expert Mixture-of-Experts model, to a single serving-grade ~5 GB, 8-layer slice that runs on a 2019 Lenovo ThinkPad L390 laptop (Intel Core i5-8265U, 7.4 GB usable single-channel DDR4, n...