Eye movement analysis for context inference and cognitive-awareness: wearable sensing and activity recognition using electrooculography
Andreas Bulling · Repository for Publications and Research Data (ETH Zurich) · 2010
Context-awareness has emerged as a key area of research in mobile and ubiquitous computing.Context-aware systems aim to sense the user's context and use this information to proactively adapt their behaviour to the user's personal needs.The context of a person is usually defined as a combination of different personal and environmental factors.Among these factors, physical activity is widely considered to be one of the most important contextual cues.Accordingly, machine recognition of human physical activity has attracted considerable research interests in recent years.Advances in activity recognition were traditionally achieved by using modalities such as body posture and gestures, interactions between people, or by combining on-body and ambient sensors.Despite these advances, activity recognition systems that use such modalities are limited with respect to the types and complexity of activities they are able to detect.In addition, activity recognition using more subtle cues, such as user attention or intention, or recognition that takes into account cognitive aspects of a user's activity, such as the experience or task engagement, remains largely unexplored.The current work introduces eye movement analysis as a novel modality for context inference and cognitive-awareness.The movements our eyes perform as we carry out different activities reveal much about the activities themselves.In the same manner, location or a particular environment influence our eye movements.Finally, unconscious eye movements are strongly linked to a number of cognitive processes of visual perception.This link to cognition makes eye movements a distinct source of information on a person's context beyond physical activity and location.Eventually, information derived from eye movements may allow us to extend the current notion of context with a cognitive dimension, leading to so-called cognitive-aware systems.This thesis comprises six scientific publications that address five aims of the work: (1) to introduce eye movement analysis as a modality for contextawareness and activity recognition as well as a trailblazer for cognitive-aware systems; (2) to develop a wearable and self-contained eye tracker for longterm eye movement recordings based on electrooculography (EOG); (3) to investigate eye gestures continuously detected from EOG signals for real-time eye-based human-computer interaction; (4) to develop an architecture for eyebased activity recognition comprising eye movement features and algorithms for EOG signal processing and eye movement analysis; and (5) to recognise daily life activities from eye movement data in natural mobile and stationary settings using this architecture.This work developed the so-called wearable EOG goggles, an embedded eye tracker that -in contrast to common systems using video -relies on EOG.Challenges associated with wearability, eye movement analysis, and signal artefacts caused by physical activity were addressed with a combination of a light-weight mechanical design, robust algorithms for eye movement detection, and adaptive EOG signal processing.The final system features real-