Dense FPGA Compute Using Signed Byte Tuples
Martin Langhammer, Simon Finn, Sergey Gribok, Bogdan Pasca · 2021
The importance of AI to FPGA has resulted in ever increasing low precision hard arithmetic features in newer devices. Many FPGAs, including those from Achronix, Intel, and Xilinx, have significantly increased the density of INT8 and INT9 embedded multipliers. Mainstream devices with these enhanced densities still support the traditional intermediate integer (typically 18-bit) multipliers, with IEEE-754 floating-point now becoming more prevalent as well.Recently, Intel introduced the Stratix 10 NX FPGA, which is targeted specifically at AI acceleration. This device contains a new type of AI-specific DSP Block with approximately an order of magnitude higher INT8 density than previous FPGA industry DSP Blocks. Larger standard FPGA integer precisions, however, are not directly supported. Intel has described some methods of aggregating larger multipliers from the NX Blocks, but these are somewhat smaller than typically used by DSP applications. Larger multiplications can also be useful for other AI applications, such as found in training. In this paper, we introduce the concept of signed tuples, which can be used to assemble signed multipliers into more useful larger precision multipliers by leveraging FPGA soft-logic inexpensively. We demonstrate several constructions of INT16 multipliers, with some modes requiring less than 3 ALMs per INT16 multiplier when implemented in a tensor format. We also describe the application of these methods to even larger multipliers and alternate constructs such as complex multiplication. We show that there is essentially no performance degradation or system fitting impact from our method. The mid-size NX device can support up 33 TOPs INT16 (from 29,700 constructed INT16 multipliers on a mid-speed grade device) with this approach, which is higher than any other current or announced monolithic die FPGA. Our methods are not limited to FPGA, or any particular starting precision, and so may be used for other aggregations as well.