Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech
Özge Alaçam · DSpace repository (University of Tartu) · 2019
Cross-modality between vision and language is a key component for effective and efficient communication, and human language processing mechanism successfully integrates information from various modalities to extract the intended meaning.However, incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges for a successful communication.In that case, incompleteness in one channel can be compensated by information from another one (if available).In this paper, by employing a visualworld paradigm experiment, we investigated the dynamics between syntactically possible gap fillers for incomplete German sentences and the visual arrangements and their effect on overall sentence interpretation.