FloatAP: Supporting High-Performance Floating-Point Arithmetic in Associative Processors
Kailin Yang, José F. Martínez · 2024
Associative Processors (AP) enable in-situ, data-parallel computation in content-addressable memories (CAM). In particular, arithmetic operations are accomplished via bit-serial sequences of bulk search and update primitives. The data-parallel nature of AP-based computation, whereby thousands of operations can take place concurrently, more than makes up for the slowness of its bit-serial arithmetic; as a result, APs can deliver very high performance. Indeed, recent AP proposals have shown promising results across various application domains, primarily for integer codes. However, floating-point (FP) support is crucial for many critical AI/ML and scientific workloads. This paper introduces FloatAP, the first highly programmable AP architecture tailored for high-performance FP arithmetic. We show that the straightforward application of AP-based bit-serial arithmetic to FP leads to insufficient computational through-put. With FloatAP, we propose a bit- and component-parallel vector mantissa alignment method that significantly improves the throughput of vector FP addition and reduction. We also repurpose the existing adder trees in modern APs to efficiently implement multiplication and dot product operations, markedly outperforming AP's traditional bit-serial multiplication. FloatAP is flexible enough to accommodate different FP formats while maintaining high utilization and integer compat-ibility. It is programmable using RISC-V ‘V’ vector extension and easily accommodates domain-specific languages and ML frameworks. We evaluate FloatAP using contemporary ML workloads, including CNN-, RNN-, and LLM-based parallel codes. The results show that, on average, FloatAP achieves 2.7 × higher inference throughput compared to an area-equivalent NVIDIA A100 Tensor Core GPU for single-precision FP (1.5 × for bfloat16) and 2.4 × higher energy efficiency (1.6 ×for bfloat16).