Learning by Stretching Deep Networks
Gaurav Shankar Pandey, Ambedkar Dukkipati · 2014
In recent years, deep architectures have gained a lot of prominence for learning complex AI tasks because of their capability to incorporate com-plex variations in data within the model. How-ever, these models often need to be trained for a long time in order to obtain good results. In this paper, we propose a technique, called ‘stretch-ing’, that allows the same models to perform considerably better with very little training. We show that learning can be done tractably, even when the weight matrix is stretched to infinity, for some specific models. We also study tractable algorithms for implementing stretching in deep convolutional architectures in an iterative man-ner and derive bounds for its convergence. Our experimental results suggest that the proposed stretched deep convolutional networks are capa-ble of achieving good performance for many ob-ject recognition tasks. More importantly, for a fixed network architecture, one can achieve much better accuracy using stretching rather than learn-ing the weights using backpropagation. 1.