Hierarchical Multi-Label Classification: Going Beyond Generalization Trees
Peerapon Vateekul · 2012
of a dissertation at the University of Miami. Dissertation supervised by Professor Miroslav Kubat. No. of pages in text. (183) Traditional computational approach to automated classification assumes that each object should be assigned to only one out of two or more classes. However, some realworld applications digress from this generic scenario in two important ways. First, each example can belong to several classes simultaneously (multi-label classification). Second, the classes can be hierarchically ordered in the sense that some are more specific versions of others (hierarchical classification). Seeking to address both of these issues, the presented work deals with “hierarchical multi-label classification.” The task has recently received considerable attention; databases in various fields, including web repositories, digital libraries, or genomics, are known to be organized as hierarchies. Seeking to start with something relatively simple, scientists have focused on the special case where the inter-class relations are captured by tree-structured hierarchy. This, however, is not enough. Very often, some classes have more than one parent, in which case the mutual relations (if they are known) have to be described by a hierarchy structured as a directed acyclic graph (DAG). This dissertation intends to contribute to this more general problem. Literature survey indicates that, in non-hierarchical multi-label classification, good performance is achieved when a Support Vector Machine (SVM) is used to induce each class separately. This said, some experiments suggest that further improvement can be achieved by explicitly dealing with the problem of imbalanced training sets because, in most classes, negative examples heavily outnumber positive ones. The author proposes a solution in terms of a technique referred to as R-SVM; the idea is to re-adjust the SVM-hyperplane offset accordingly. Experiments in the first part of this dissertation rely on data from domains of text-categorization. More important, however, is then the second part that focuses on hierarchical multi-label classification. Here, the author proposes a new technique, HR-SVM, that essentially constitutes a hierarchical extension of R-SVM proceeding in a top-down fashion from more general to more specific classifiers. The weakness of this approach is known as “error propagation”: examples misclassified at higher levels are propagated down the hierarchy, thus resulting in negative performances at the lower levels. HR-SVM contains a mechanism to correct this kind of errors. The system has been subjected to extensive experiments with many domains from the field of gene function prediction. The results show that the new technique compares favorably with other existing approaches along various performance criteria. To my beloved parents