Alleviating the complexity of the Combinatorial Neural Model using a committee machine
Hércules Antônio do Prado, Karla F. Machado, Paulo Martins Engel · International Conference on Data Mining · 2000
Knowledge Discovery from Databases (KDD) can be seen as a set of computer-aided knowledge discovery techniques scaled up to very large databases. By this way, the old process of discovery has experienced amazing improvements by: (a) allowing well-known Machine Learning and Statistical algorithms run for larger data sets with good performance; and (b) making easier tasks like data gathering and cleansing, parameter and model selection, and so on. In this paper we take the Combinatorial Neural Model (CNM), proposed by Machado and Rocha ([6], [7], and [8]), and explore the adoption of a committee machine to cope with the complexity problem present in this model. In our proposal, a static committee machine is built on a certain number of CNMs. The committee machine takes the discrete outputs from the individual models, and proceeds by choosing the most accurate output using a voting process. Many works has been presented to alleviate the complexity problem in CNM and this one represents an alternative that can be combined with other solutions. The results from the experiments point out that, by means of a committee machine, one can reach a better trade-off between predictive accuracy and complexity. To evaluate the effectiveness of the technique, we apply it to some well-known data sets from the UCI repository [2]. Finally, it is suggested some extensions to improve the committee machine in order to obtain a better generalization performance, towards a universal approximator.