Attention, Movie Cuts, and Natural Vision: A Functional Perspective
Laurent Itti, Ran Carmi · 2008
Paying attention to the right thing at the right time underlies the ability of humans and other animals to learn, perceive, and interact with their environment. Attentional selection enables biological organisms with limited neural resources to extract the most pertinent information from the barrage of sensory inputs that they normally experience in the real world. The goal of the research described in this dissertation is to characterize the output, input, and associated computational transformations that underlie attentional selection during natural vision. Such functional understanding of attentional selection would help build more intelligent machines that could behave autonomously in real world environments. It could also lead to better diagnostic tools and treatments for medical conditions, such as Autism, ADHD, and Parkinson Disease, which are characterized by anomalous patterns of attentional selection. The following key questions are addressed: What do movies reveal about the nature of sensory inputs to the human visual system? Which sensory inputs causally attract attention and how do they compare to each other? How is attentional selection affected by interactions between past and present sensory inputs? To answer these questions, the eyes of human observers were tracked non-intrusively as they watched either continuous or MTV-style (discontinuous) video clips. The main conclusions of this dissertation are based on a series of quantitative analyses, in which human gaze behavior and the prediction accuracy of related computational models were compared across viewing conditions, space, and time. The results indicate that sensory inputs are naturally discontinuous, and that the human visual system can keep the mind informed of the most pertinent information at every point in time without requiring either visual or mental continuity. It is also shown that dynamic visual correlates of attentional selection (e.g., motion contrasts) play a dominant causal role in attracting attention during natural vision. In comparison, some static visual correlates (e.g., color contrasts) play a relatively weaker causal role, whereas others (e.g., orientation contrast) are non causal correlates, potentially reflecting top-down causes. Lastly, it is demonstrated that perceptual memory is utilized for guiding attention across several gaze shits when persistent visual context is available.