The Importance of Cross Database Evaluation in Sound Classification
Arie Livshin, Xavier Rodet · 2003
In numerous articles (Martin and Kim, 1998; Fraser and Fujinaga, 1999; and many others) sound classification algorithms are evaluated using "self classification "- the learning and test groups are randomly selected out of the same sound database. We will show that "self classification " is not necessarily a good statistic for the ability of a classification algorithm to learn, generalize or classify well. We introduce the alternative "Minus-1 DB " evaluation method and demonstrate that it does not have the shortcomings of "self classification". 1 Testing Platform The importance of cross database evaluation will be demonstrated through a variety of classification experiments. 1.1 The Test Set The Sounds. In order to demonstrate well the claims in the paper, we extracted out of 5 sound databases, recorded in various acoustic conditions and different equipment, the samples of 7 instruments common to them, played with a "standard " playing technique. The instruments are: Bassoon,