Two Level Multimodal Fusion Algorithm for Semantic Video Analysis
Wei Wei · 2007
To extract video semantic concepts combing different modalities,a two level multimodal fusion method for video semantic concept analysis is proposed.Multimodal features of video are divided into several groups.The fact that each group of has distinct features,a hierarchical hidden Markov models(HHMM) is constructed for the purpose of first-level fusion.Then outputs of first-level fusion are combined using a kernel function,by which a hyper-plane with better classification for video semantic concept is obtained.The results of experiments comparing to other fusion methods support that the two-level fusion method utilizes different modal feature in semantic concept analysis,and could effectively combine multimodal features.