Overcoming Memory Limitations for On-Device AI and LLM in Wearable AR Systems
Huichu Liu, Daniel H. Morris, Lita Yang, Ekin Sumbul, Tony F. Wu, Jaspreet Gandhi, Camillo Tamma, Umut Arslan, Baolin Yi, Rawan Naous, Paul S. Diefenbaugh, Édith Beigné · 2024
Wearable augmented reality (AR) devices will create new AI-enabled user experiences; but realization of this opportunity is challenged by the power and size of memory technologies. We seek to overcome these limitations. Low energy 3D interconnects enable integration of either SRAM or DRAM chiplets in compact packages. A wide interface of 33K and 0.9K connections is achieved, respectively. Moreover, cooptimizing memory for workload characteristics, reduces component energy >47% for AI workloads. These memories, along with emerging compute-in-memory subsystems, are evaluated in systems to establish the opportunities and challenges of these technologies for on-device AI in wearable AR systems.