Auto-encoder using the bi-firing activation function
Zihong Cao, Guangjun Zeng, Wing W. Y. Ng, Jincheng Le · 2014
Training the whole deep neural network together is restricted by the gradient diffusion problem. Greedy layer-wise training of an auto-encoder has achieved promising results in deep neural networks. However, it can not learn useful input representation from the original input directly. In this work, we propose to use the bi-firing activation function for auto-encoder with an end-to-end training scheme. It not only improves the training efficiency but also learns better features than the traditional stacked auto-encoder. Experimental results show that it extracts more representative features and also outperforms the stacked auto-encoder in supervised classification task.