F-Measure Based End-to-End Optimization of Neural Network Keyword Detectors
Tomohiro Tanaka, Takahiro Shinozaki · 2018
F-measure is a widely used evaluation criterion for keyword detection where measures such as accuracy and cross-entropy are not appropriate since the numbers of positive and negative samples are largely unbalanced. In this work, we propose a soft decision version of the F-measure as an objective function to train end-to-end neural network based keyword detectors. It has advantages that the training objective is consistent with the final evaluation measure. We apply it to acoustic embedding based keyword detectors using long short term memory (LSTM). Evaluation experiments using the WSJ corpus show that the proposed F-measure based training improves keyword detection performance than cross-entropy training while eliminating a meta-parameter to balance the effect of positive and negative samples in training. Additionally, we propose end-to-end continuous dynamic programming (DP) matching using a two-dimensional recurrent neural network (2D-RNN) and train it with the proposed F-measure criterion. While it has unstable behavior in the training, higher detection performance than the embedding method has achieved.