Tutorial: Generative AI on Edge Devices: Models, Hardware and Systems
Paul N. Whatmough · 2025
Generative AI models have introduced unprecedented new capabilities to edge devices, everything from writing poetry to summarizing complex documents, generating incredible images and even video clips. However, this new breed of neural networks, spanning large language, vision and multi-modal models, poses numerous challenges for edge devices, stressing both compute and memory-system performance and driving active research across many disciplines. In this tutorial, I will discuss the latest Generative AI networks and the challenges they present for the hardware system, along with the latest network/hardware co-design opportunities to help close the power and performance gap for edge-device AI applications.