Energy-Efficient, Mobile Computer Vision and Machine Learning Processors
Ziyun Li · Deep Blue (University of Michigan) · 2019
Technology scaling has driven computing devices to be faster, cheaper, and smaller while consuming less power in past decades. However, as technology scaling has become increasingly difficult in recent years, power has become the major constraint in performance, and thus, the improvement in the performance of mobile devices has begun to diminish. Moreover, emerging intelligent mobile systems are demanding increasing computing power. In light of this challenge associated with artificial intelligence, domain-specific architectures are widely believed to be the path to realizing considerable improvements in the efficiency, performance and cost of intelligent mobile systems. This thesis presents several algorithm, architecture and circuit co-optimized solutions for intelligent and autonomous mobile systems, including vision-based stereo depth, optical flow, simultaneous localization and mapping (SLAM) and convolutional neural network- (CNN)-based image recognition. Four prototypes are implemented for demonstration and verification. The first two prototypes include a depth estimation processor and a 6D vision processor that enable real-time dense depth and motion perception, respectively. The third prototype is a CNN-SLAM processor that estimates ego-motion for vision-based navigation. Together, these prototypes form a geometric understanding of the environment for mobile systems. The fourth prototype is an ReRAM-CNN processor that enables semantic understanding through machine learning. The work presented in this dissertation exploits various optimizations including parallelism, scheduling, exploiting sparsity and circuit customization to overcome the complexity of these problems for extremely energy-efficient, real-time, robust operation. The impact is significant in the age of AI as mobile systems can become increasingly intelligent in daily life, powered by these proposed solutions.