Zhejiang Business Technology Institute

SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation

Large language models (LLMs), particularly decoder-only models with auto-regressive decoding, have achieved remarkable performance across a wide range of tasks, but their inference efficiency remains a critical bottleneck due to rapidly growing parameters and memory-bound generat...