Comprehensibility of Classification Trees – Survey Design Validation
Rok Piltaver, Mitja Luštrek, Matjaž Gams, Sanda Martinčić-Ipšić · 2014
Abstract: Classifier comprehensibility is a decisive factor for practical classifier applications; however it is ill-defined and hence difficult to measure. Most algorithms use comprehensibility metrics based on classifier complexity – such as the number of leaves in a classification tree – despite evidence that they do not correspond to comprehensibility well. A classifier comprehensibility survey was therefore designed in order to derive exhaustive comprehensibility metrics better reflecting the human sense of classifier comprehensibility. This paper presents an implementation of a classification-tree comprehensibility survey based on the suggested design and empirically verifies the assumptions on which the survey design is based: the chosen respondent performance metrics measured while solving the chosen tasks can be used to indirectly but objectively measure the influence of chosen tree properties on their comprehensibility.