Cascaded Mutual Modulation for Visual Reasoning
Yiqun Yao, Jiaming Xu, Feng Wang, Bo Xu · 2018
Visual reasoning is a special visual question answering problem that is multi-step and compositional by nature, and also requires intensive text-vision interactions.We propose CMM: Cascaded Mutual Modulation as a novel end-to-end visual reasoning model.CMM includes a multi-step comprehension process for both question and image.In each step, we use a Feature-wise Linear Modulation (FiLM) technique to enable textual/visual pipeline to mutually control each other.Experiments show that CMM significantly outperforms most related models, and reach stateof-the-arts on two visual reasoning benchmarks: CLEVR and NLVR, collected from both synthetic and natural languages.Ablation studies confirm that both our multistep framework and our visual-guided language modulation are critical to the task.Our code is available at https://github. com/FlamingHorizon/CMM-VR.