A gradient descending solution to the LASSO criteria
Nan Zhang, Shuqing Zeng · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · 2006
In this paper, we propose a new perspective to achieve sparseness via the winner-take-all principle for the linear kernel regression and classification tasks. We form the duality of the LASSO criteria, and transfer an /spl lscr//sub 1/ norm minimization to an /spl lscr//sub /spl infin// norm maximization problem. We introduce a novel winner-take-all neural network solution derived from gradient descending, which links the sparse representation and the competitive learning scheme. This scheme is a form of unsupervised learning in which each input pattern comes through learning, to be associated with the activity of one or at most a few neurons. However, the lateral interaction between neurons in the same layer is strictly preemptive in this model. This framework is applicable to a variety of problems, such as independent component analysis (ICA), feature selection, and data clustering.