Zhaoyu Lu

SATIN: a sparse attention application-specific instruction-set processor with custom mixed-precision extensions for energy-efficient edge inference of transformer large language models

Edge inference of Transformer large language models is constrained by the quadratic cost of self-attention and by the memory traffic of the key-value cache. Existing accelerators treat sparsity and mixed precision as separate fixed-function mechanisms and provide no accuracy guar...