See, Hear, and Learn to Describe--Crossmodal Information Processing Opens the Way to Smarter AI
Kunio Kashino · NTT technical review · 2019
At NTT Communication Science Laboratories, we are researching information processing across different types of media information such as images, sounds, and text.This is known as crossmodal information processing.The point of crossmodal information processing is to create a common space, which is a place where multiple types of media data are associated.The common space enables us to realize new functions that have never existed before: new transformations between different media-such as creating images and descriptions from sound-and the acquisition of concepts contained in media information.