An Energy-based Model for Feature Selection
Hugo Jair Escalante, Manuel Montes, Luis Enrique Sucar · 2008
In this paper we propose an energy-based model (EBM ) for selecting subsets of features that are both causally and predictively relevant for classication tasks. The proposed method is tested in the causality challenge, a competition that promotes research on strengthen feature selection by taking into account causal information of features. Under the proposed approach, an energy value is assigned to every conguration of features and the problem is reduced to that of nding the conguration that minimizes an energy function. We propose an energy function that takes into account causal, predictive, and relevance/correlation information of features. Particularly, we introduce potentials that combine the rankings of individual feature selection methods, Markov blanket information and predictive performance estimations. The conguration with lower energy will be that oering the best tradeo between these sources of information. Experimental results show that despite being simple, the EBM approach is able to select highly predictive features. In particular, the combined score of feature relevance and the predictive estimation resulted very useful.