Fine-Grained Fusion: The Missing Piece in Area-Efficient State Space Model Acceleration
Robin Geens, Arne Symons, Marian Verhelst · 2025
State Space Models (SSMs) offer a promising alternative to transformers for long-sequence processing. However, their efficiency remains hindered by memory-bound operations, particularly in the prefill stage. While MARCA, a recent first effort to accelerate SSMs through a dedicated hardware accelerator, achieves a great speedup over high-end GPUs, an analysis of the broader accelerator design space is lacking. This work systematically analyzes SSM acceleration opportunities from both the scheduling perspective, through fine-grained operator fusion, and the hardware perspective, through design space exploration, using an extended version of the Stream modeling framework. Our results demonstrate that the improved data locality stemming from our optimized fusion and scheduling strategy enables a speedup of up to $4.8 \times$ over unfused execution, while our adaptive memory-aware fusion approach reduces on-chip memory requirements by an order of magnitude without sacrificing performance. We further explore accelerator design trade-offs, showing that a fusion-aware hardware architecture can achieve $1.78 \times$ higher performance than the state-of-the-art MARCA accelerator, within the same area budget. These results establish operator fusion as a key enabler for next-generation SSM accelerators.ACM Reference Format:Robin Geens, Arne Symons, and Marian Verhelst. 2025. Fine-Grained Fusion: The Missing Piece in Area-Efficient State Space Model Acceleration. In Proceedings of (PACT ’25). ACM, New York, NY, USA, 11 pages. https: //doi.org/XXXXXXX.XXXXXXX