Design and Analysis of Twin Tower High Bandwidth Memory (HBM) Architecture for Large Memory Capacity and High Bandwidth System
Taesoo Kim, Jiwon Yoon, Seonguk Choi, Haeyeon Kim, Haeseok Suh, Hyunjun An, Jungmin Ahn, Hyunah Park, Joungho Kim · 2024
The rapid advancement of AI, driven by generative AI based on transformer models, demands memory systems with higher capacity and bandwidth, yet current HBM technologies face scaling limitations due to architectural and manufacturing constraints. To address this, we propose the twin tower HBM architecture, integrating two DRAM stacks on a single, elongated base die to double memory capacity and enhance bandwidth. The extended base die also allows for architectural flexibility, including the integration of near-memory computing units. Supporting up to 12 DRAM stacks per GPU, this architecture achieves 576 GB capacity and 1,638 GB/s bandwidth—a 27.9% improvement over HBM3e—while maintaining robust signal integrity through optimized interposer channel designs validated via EM and SPICE simulations. This innovative solution provides a cost-effective path to scaling AI infrastructure and overcoming memory bottlenecks in next-generation workloads.