A Cache Modeling Framework for Accelerator Systems

Abstract

Modern hardware accelerators increasingly rely on cache-based memory systems to improve modularity and tolerate irregular memory behavior in sparse and data-dependent workloads. However, configuring accelerator caches remains challenging: designers must choose cache capacity, Miss Status Holding Register (MSHR) count, subentry provisioning, and fill bandwidth without a fast model that captures the timing-dependent behavior of non-blocking caches. Existing analytical models often focus on cache residency or miss counts, while cycle-level simulation captures miss merging and resource contention at substantially higher cost. This paper presents PlastiCache, a timing-augmented reuse-distance model for non-blocking accelerator caches. PlastiCache represents an accelerator memory stream as address–time pairs and extends reuse-distance analysis with predicted service intervals for outstanding misses. This formulation classifies each request as a cache-resident hit, an in-flight MSHR merge, or a new off-chip miss, while incorporating miss-handling capacity, memory-service timing, and return-path bandwidth as event constraints over the request stream. As a result, PlastiCache identifies whether a cache design is limited by cache capacity, MSHR availability, subentry pressure, DRAM service, or fill bandwidth. PlastiCache targets early-stage cache design-space exploration by preserving the relative ordering of candidate configurations. Across broad sweeps of workloads and cache designs, PlastiCache achieves a mean Spearman ranking correlation of 0.93 with RTL on latency–area tradeoff curves, while achieving geometric-mean speedups of 545 × over tree-based analytical models and 303 × over Verilator-based RTL simulation. We further integrate PlastiCache into an end-to-end accelerator cache sizing flow with a Chisel-based sparse matrix multiplication processing element and a synthesizable cache generator. FPGA prototyping across real-world workloads, including SuiteSparse matrices and a large language model, shows that the PlastiCache-Balanced design reduces delay–area product by 45.6% and 25.9% on average compared with MOMS and MiCache, respectively, with maximum reductions of 73.8% and 37.3%.