Exploiting Multimodal Context in Image Retrieval
Rohini K. Srihari, Zhongfei Mark Zhang · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · 1999
THISRESEARCH EXPLORES THE INTERACTION of textual and photographicinformation in multimodal documents.The World Wide Web (WWW) may be viewed as the ultimate, large-scale, dynamically changing, multimedia database.Finding useful information from the WWW without encountering numerous false positives (the current case) poses a challenge to multimedia information retrieval systems (MMIR).The fact that images do not appear in isolation, but rather with accompanying collateral text, is exploited.Taken independently, existing techniques for picture retrieval using collateral text-based methods and image-based methods have several limitations.Text-based methods, while very powerful in matching context, do not have access to image content.Image-based methods compute general similarity between images and provide limited semantics.This research focuses on improving precision and recall in an MMIR system by interactively combining text processing with image processing (IP) in both the indexing and retrieval phases.A picture search engine is demonstrated as an application.