Hardware-Software Co-Design for Power-Efficient Edge-AI Systems
Karthik Wali · Journal of Artificial Intelligence Machine Learning and Data Science · 2024
The proliferation of intelligent applications running at the network edge-everything from smart cameras and industrial IoT sensors to self-driving drones-has driven the need for high-performance but low-power edge-AI systems.In contrast to inference in the cloud, edge-AI has to deal with harsh constraints on energy, latency, and computational capacity, which requires a fundamental shift away from traditional isolated hardware or software-only optimization methodologies.This article explores the hardware-software co-design paradigm as an integrated approach to solving the multiple dimensioned challenges in the design of power-effective edge-AI systems.Hardware-software co-design entails concurrent and synergistic optimization of system architecture and the software stack.By closing the formerly distinct spaces of hardware design (e.g., AI accelerators, memory stacks, power control units) and software engineering (e.g., neural network structure, compilers, scheduling algorithms), this method targets deriving globally optimal solutions specific to edge use cases.The paper begins by contextualizing the evolution of edge-AI, identifying its unique constraints-such as real-time processing requirements, energy autonomy, limited thermal envelopes, and increasing model complexity-and explaining why conventional design approaches fall short.We then perform a comprehensive literature review that synthesizes recent breakthroughs in co-designed edge systems.Eminent techniques involve integration of sparsified and quantized deep learning models with low-power tensor processing units (TPUs), the utilization of dynamic voltage and frequency scaling (DVFS) with real-time operating systems (RTOS), and co-optimization environments that dynamically change model complexity according to runtime power profiles.We also discuss tools and middleware-like TensorRT, Apache TVM, and Xilinx Vitis AI-that support hardware-conscious model compilation and runtime adaptability.