Action Sequence Recognition in Videos by Combining a CTC Network with a Statistical Language Model
Mengxi Lin, 中順 井上, Nakamasa Inoue, 浩一 篠田, Koichi Shinoda · Institutional Repositories DataBase (IRDB) · 2017
Action sequence recognition aims to recognize what actions occur in a video and their temporal order. In this paper, we propose to combine an LSTM network trained with Connectionist Temporal Classification (CTC) with a statistical language model for action sequence recognition. The statistical language model captures the relations between action instances, which are hardly learned by the CTC network. Our experiments on the Breakfast dataset show that the statistical language model can significantly boost the recognition accuracy of the CTC network, from 37.0% to 43.4%.