Alpamys Makazhan

Exact Layer Streaming: LoRA Fine-Tuning of an 8B Model on a 4 GB Laptop GPU

Parameter-efficient fine-tuning is bounded by a hard constraint: the frozen base model must fit in GPU memory. Layer streaming — holding a small pool of decoder layers in VRAM and fetching the rest from host RAM on demand — removes that constraint in principle, but the published...