A hybrid learning system with a hierarchical architecture for pattern classification
D. Atukorale · The University of Queensland · 2002
This thesis addresses the use of hybrid learning systems that combine supervisednand unsupervised learning methods for pattern classification. Most neural networknresearch applied to pattern recognition has been focussed on supervisednlearning, and network models like the MLP provide an efficient method to designnan arbitrarily complex non-linear classifier. However, there are some problemndomains that are not solved in a satisfactory way by means of a single classifier.nWhen the abstraction level of the classification task increases, the shape of thendecision regions can become very complex, requiring impossibly large amountsnof training data to form the class boundaries. This problem can be alleviatednby using unsupervised learning techniques to reduce the number of degrees ofnfreedom in the data. Hybrid learning systems which combine supervised andnunsupervised learning methods have been very popular in this regard.nnnnnn This thesis introduces a novel hybrid system with a hierarchical architecturenwhich is based on the neural gas (NG) algorithm for pattern recognition problems.nThe NG algorithm in the proposed learning system uses a much fasternvariation of the original NG algorithm by reducing the time complexity of itsnsequential implementation. The computationally expensive part of the adaptationnstep of the original NG algorithm is the determination of the neighborhoodnranking. This requires an explicit ordering of all distances between the referencenvectors and the input pattern, and this has time complexity 0(N log N). Thisnproblem is addressed here by introducing an implicit ranking method which reducesnthe time complexity to 0(N).nnnnn nThe proposed learning system generates multiple classifications for every datanpattern presented, and these are registered as qconfidence valuesq. The mostnsuitable functional form for calculating confidence values was determined empiricallynand it smoothly assigns confidence values from 1 to 0. These confidencenvalues allow the system to employ a variety of classifier fusion techniques to combinenindividual classifications to produce the predicted class for a pattern. Fourndifferent classifier combination techniques were used in the comparisons. It wasnshown that combining a network performance measure with confidence valuesnby means of the fuzzy integral leads to the best classification performance. Thenperformance of the proposed system was compared with that of other techniquesnon three well-known benchmark data sets, and promising results were obtained.nnnnn nFinally, it was shown that the boosting algorithm can be applied to a learningnsystem that uses mixed supervised/unsupervised methods. The boosted learningnsystem gave improved results over those obtained without boosting.n