LoRA Fine-Tuning Enables Controllable Mathematical Reasoning in Small Language Models: A Gating Pipeline on Qwen3.5-0.8B

Abstract

We design a gating pipeline for LoRA fine-tuning of Qwen3.5-0.8B, a 24-layer hybrid of GatedDeltaNet and Full Attention, executed entirely on CPU. The pipeline runs three ablations: layer ablation selects full-module joint fine-tuning over single-module configurations; data curriculum: AllOnce matches easy-first and is selected by the simplicity rule; LoRA rank ablation shows rank 8 matches rank 16 at small scale and outperforms it at larger scale. The gated model preserves reasoning accuracy (93% retention on int500, 99% on the showcase set) while transforming output controllability on int500: boxed usage rises from 0% to 61.2%, automatic accuracy from 0% to 12.2%, average length falls from 985 to 265 tokens (−73%), and truncation drops from 12.0% to 0%.