Real-time Framework for On- and Off-line Multimodal Human-Human and Human-Robot Interaction
Frank Wallhoff, Jürgen Gast, Alexander Bannat, Stefan Schwärzler, Gerhard Rigoll, Cornelia S. Wendt, Sabrina Schmidt, Michael Popp, Berthold Färber · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · 2008
All authors contributed equally. Abstract — In this paper we present a framework for realtime processing of multimodal data, which can be used for on- and off-line processing of perceived data in interactions. We propose the use of a framework based on the Real-time Database (RTDB). This framework allows easy integration of input and output modules and thereby concentrating on the core functionality of the module. Furthermore the asynchronous data from different sources is synchronized and can be recorded for off-line processing. This recorded data can be used to train recognition modules, which can then be used again on-line. Experiments and first results with off-line human-human and on-line human-robots are reported. I.