Accelerating Recommendation Systems With In-Memory Embedding Operations
Mengyuan Li, Dayane Alfenas Reis, Ann Franchesca Laguna, Michael Thaddeus Niemier, Xiaobo Sharon Hu · IEEE transactions on circuits and systems for artificial intelligence. · 2024
Recommendation systems (RecSys) can effectively suggest items to a given user by predicting their preferences based on the massive amount of historical data from a collection of users. RecSysbased on deep neural networks (DNNs) are widely adopted, and must handle large embedding tables (ETs) and many ET related operations. The memory size and bandwidth of conventional computing architectures usually restrict the performance of RecSys, especially for operations related to ETs. To reduce both memory and computational bottlenecks, this work proposes implementing compressed ETs on an in-memory-computing (IMC) architecture, nMARS, for accelerating DNN-based RecSys. The IMC fabric consists of random-access memories (RAMs), configurable random-access/content-addressable memories (RA/CAMs), and crossbars that can be implemented with CMOS or ferroelectric field effect transistors (FeFETs). RA/CAMs can be configured to switch between a content addressable memory mode and RAM mode to support RecSys operations. Based on this IMC fabric, we design IMC-friendly ETs and explore the integration of mixed-dimension and mixed-precision ET compression techniques, which reduce the memory requirements, through a mapping scheme specifically designed to support the compressed format. Detailed circuit-level and system-level evaluations show that nMARS designed with a CMOS 45 nm process node achieves$\boldsymbol{\sim}$32$\boldsymbol{\times}$and 14$\boldsymbol{\times}$latency improvement on the filtering and ranking stages of the MovieLens dataset compared to a 16 nm GPU-based solution. Moreover, for ET related operations on the Criteo dataset, nMAR Scan achieve an 80$\boldsymbol{\times}$speedup compared to the GPU-based solution. For a FeFET implementation, it achieves a 39% energy reduction compared to the previous FeFET-based in-memory RecSyssolution (Li et al., 2022) on the MovieLens dataset.