A Probabilistic Approach to Text Generation of Human Motions extracted from Kinect Videos
Mizuki Kobayashi, Ichiro Kobayashi, Hideki Asoh, Sergio Guadarrama · 2013
In this study, we propose a framework for proba- bilistic text generation of human motions extracted from Kinect videos. We capture human motions by a Kinect camera and extract the time-series data of the motions from the videos. The time-series data are applied by several dimension reduction procedures and then turned to be the form which can be applied to machine learning. A pair of the analyzed time-series data and its intermediate representation which corresponds to the semantics of the human motion is learned by a log-linear model. As linguistic resources to generate a text, we collected various natural language expressions for human motions and build a bi-gram model for each motion. In our framework, once the intermediate representation is decided by observing time-series data; a proper bi-gram model corresponding to the intermediate representation is chosen; and then a text is generated by solving dynamic programming of the bi-gram model. Through experiments to generate texts describing human motions, we have confirmed that our proposed framework works well. Index Terms—probabilistic text generation, bi-gram, Sym- bolic Aggregation approXimaion(SAX), time-series data, Kinect