A spatial-temporal visual mid-level ontology for GIF sentiment analysis
Zheng Cai, Donglin Cao, Dazhen Lin, Rongrong Ji · 2016
With the progress of social medias, an increasing number of dynamic multimedia, such as video clips (GIF), were used in Internet. Many of them show the subjective sentiment of users. GIF sentiment analysis is important for political election predication and economic indicator evaluation. Unfortunately, such task is quite challenging because the users' sentiment hinges on spatial-temporal visual concepts. And the relationship between such concepts and overall sentiment polarity remains unknown. In this paper, dedicated to build a bridge to exploring such relationship, we proposed a SentiPair Sequence based spatial-temporal visual sentiment ontology. The ontology serves as a mid-level representation of GIF sentiment analysis. In order to analyze sentiment polarity, we first constructed a Synset Forest to define the semantic tree structure of visual sentiment concepts. Then, through the Synset Forest, we organically select and combine sentiment label elements to form a mid-level visual sentiment representation. Our experiments indicate that SentiPair outperforms Adjective Noun Pairs which is a state-of-art visual sentiment ontology for image. We also released our dataset (GSO-2015) to the research community. GSO-2015 contains more than 6,000 manually labeled GIFs and 40,000 unlabeled GIFs. Each is labeled with both sentiment and SentiPair Sequence.