Maxime Parker

Hidden-FFN on Apple GPUs: Compact Inference and Training from Homogeneous Sparse FFNs to WANDA-Pruned Language Models

This paper presents a first GPU implementation of Hidden-FFN (HFFN), whose companion CPU worksestablished an exact compact runtime for fixed sparse FFNs. Four Metal paths are evaluated: homogeneous FFNinference and training, then Pythia-70M, 160M, and 410M inference and training...