Deep learning of invariant spatio-temporal features from video
Bo Chen · cIRcle (University of British Columbia) · 2010
We present a novel hierarchical and distributed model for learning invariant spatio-temporal features from video. Our approach builds on previous deep learning methods and uses the Convolutional Restricted Boltzmann machine (CRBM) as a building block. Our model, called the Space-Time Deep Belief Network (ST-DBN), aggregates over both space and time in an alternating way so that higher layers capture more distant events in space and time. The model is learned in an unsupervised manner. The experiments show that it has good invariance properties, that it is well-suited for recognition tasks, and that it has reasonable generative properties that enable it to denoise video and produce spatio-temporal predictions.