Selective Combination based on Diversity-Accuracy Balance in Outlier Ensembles
Limin Shi, Chao Zhu · 2020
In the outlier detection task, unsupervised detection methods have been widely applied due to the absence of ground truth. Ensemble method is an emerging topic that has been studied recently and applied to more and more scenarios. The main challenges in this field include the selection of ensemble members and the combination of them to get final results. Most of the existing outlier ensemble methods combine all the underlying detectors without considering selection for the most appropriate ones, which may adversely affect the final result since some base detectors that are not proficient at identifying all outlier instances may be combined. Therefore, in this paper, we propose a new approach for outlier detection by taking the balance between diversity and accuracy in unsupervised ensembles into account. Specifically, clustering algorithm is adopted to induce the diversity of ensemble, and the most competent detector in each cluster is picked to improve the accuracy of ensemble, then all these selected detectors are combined to get final result. Our proposed approach is experimentally compared with other nine popular methods in the literature on fifteen real-world datasets, and the results demonstrate that the proposed method outperforms these compared methods on the majority of datasets. Moreover, our approach is also applied in a practical case for data quality control of breast cancer community cohort, and its effectiveness is further validated.