Learning autoencoders with low-rank weights
Kavya Gupta, Angshul Majumdar · 2017
In this work we propose to regularize the encoding and decoding weights of an autoencoder using low-rank penalty in the form of nuclear norm. Such a formulation models redundancy in the network. We show that our proposed method yields better classification accuracy (on an average) and denoising results than other stochastic and deterministic regularization techniques used in deep autoencoders. Our method is also considerably faster compared to these techniques. The experiments have been carried out on benchmark deep learning datasets.