Rule Weight Optimization and Feature Selection in Fuzzy Systems with Sparsity Contraints.
Edwin David Lughofer, Stefan Kindermann · European Society for Fuzzy Logic and Technology Conference · 2009
In this paper, we are dealing with a novel data-driven learning method (SparseFIS) for Takagi-Sugeno fuzzy systems, ex- tended by including rule weights. Our learning method consists of three phases: the first phase conducts a clustering process in the in- put/output feature space with iterative vector quantization. Hereby, the number of clusters = rules is pre-defined and denotes a kind of upper bound on a reasonable granularity. The second phase optimize the rule weights in the fuzzy systems with respect to least squares er- ror measure by applying a sparsity-constrained steepest descent op- timization procedure. This is done in a coherent optimization proce- dure together with elicitation of consequent parameters. Depending on the sparsity threshold, more or less rules weights can be forced towards 0, switching off some rules. In this sense, a rule selection is achieved. The third phase estimates the linear consequent parame- ters by a regularized sparsity constrained optimization procedure for each rule separately (local learning approach). Sparsity constraints are applied here in order to force linear parameters to be 0, trig- gering a feature selection mechanism per rule. In some cases, this may also yield a global feature selection, whenever the linear param- eters of some features in each rule are near 0. The method is eval- uated based on high-dimensional data from industrial processes and based on benchmark data sets from the internet and compared to well- known batch training methods in terms of accuracy and complexity of the fuzzy systems.