Learning concepts by modeling relationships
Yong Rui · 2009
Supporting multimedia search has emerged as an important research topic. There are three paradigms on the research spectrum that ranges from the least automatic to the most automatic. On the far left end, there is the pure manual labeling paradigm that labels multimedia content, e.g., images and video clips, manually with text labels and then use text search to search multimedia content indirectly. On the far right end, there is the content-based search paradigm that can be fully automatic by using low-level features from multimedia analysis. In recent years, a third paradigm emerged which is in the middle: the annotation paradigm. Once the concept models are trained, this paradigm can automatically detect/annotate concepts in unseen multimedia content. This talk looks into this annotation paradigm. Specifically, this talk argues that within the annotation paradigm, the relationship-based annotation approach outperforms other existing annotation approaches, because individual concepts are considered jointly instead of independently. We will use within concept relationship, between concept relationship and blind concept relationship to illustrate the idea.