Multilingual Multimodal Integration of Sketch and Speech: A Generic Speech Representation Model for Spatial Description
Lee-Na Teh, Alvin Wee Yeo · 2009
This paper details how multiple languages are accommodated in the multimodal integration of sketch and speech, specifically, in spatial applications. The study encompasses English, Malay, Mandarin, and two under-resourced languages in Malaysia, i.e. Melanau and Iban. The preliminary study revealed that not all spatial terms (prepositions) appear in all languages. Based on these findings, we propose a method to assist in the design and development of multilingual multimodal applications. This method employs a generic representation model for spatial description.