Short Course: Efficient Alternatives and Extensions to Deep-Learning-Based Solutions
Naveen Verma · 2018
Deep-learning systems have had profound impacts in a broad range of applications. However, it is important to remember that these represent only one class ofmachine learning. In this segment of the short course, we start by probing what the critical attributes are of deep learning, and what challenges in modeling andinference they solve. We then go on to consider the limitations of deep learning in emerging applications involving on-line learning (e.g. reinforcement learningwith embedded sensors) - namely the need for a large number of training instances and the need for very low energy. This motivates alternatives or extensions todeep learning, which make use of other forms of learning to enhance training and energy efficiency. Given the need for very low energy in many applications, weexplore how the statistical-learning can enable new hardware architectures, substantially overcoming the tradeoffs limiting conventional architectures for sensingand computation. Finally, having examined how algorithmic techniques can enhance systems, we look at how systems, and emerging technologies for sensing,can enhance algorithms. As an illustration, we consider how object-associated sensing, as enabled by IoT devices, has the potential to provide semantic structure,leading to features that can enhance the generalization of learning with simpler and easier-to-train models.