Advances in video motion analysis research for mature and emerging application areas
Adrienne Heinrich · TU/e Research Portal · 2015
Advances in video motion analysis research for mature and emerging application areasThis thesis aims at enhancing two human activities that together cover almost half of our daily routine, sleeping and watching TV.The research described in this thesis focuses on an area that potentially improves both of these otherwise rather disjunct activities.We address optimization of motion estimation algorithms for TV, and investigate the feasibility of using video motion analysis algorithms to extract valuable information about a person's sleep from video.Motion estimation (ME) for TV picture rate conversion is a rather mature application area with several decades of on-going research efforts.A number of good ME methods exist and combinations of methods with a large number of parameters are not uncommon.Objective validation of ME methods and the influence of the many parameters that are involved have become more and more important.In this thesis, a methodology has been developed that can be used for the optimization of ME methods.At the same time, the developed methodology takes into account that objective measures cannot fully model the human perception.In a case study with hierarchical 3DRS, one of the state-of-the-art ME algorithms, we explored the extensive parameter space of 13000 motion estimators and provided insights with respect to the importance and the influence of the individual parameters.We found that the motion estimators optimized with the proposed validation scheme are superior to multiple existing techniques as well as standard 3DRS with regard to performance at a low computational complexity.Although the optimization methodology uses performance measures that do not capture the full complexity of human perception, still, a good correspondence with subjectively perceived picture quality is achieved.The conducted perception test confirmed that the components of the proposed methodology are well chosen and yield motion estimators with a good picture rate conversion performance.Analyzing movements during sleep can provide a wealth of information as body movements can be associated to sleep states and sleep state transitions.Traditional sleep screening is performed in sleep clinics with polysomnography (PSG) studies, xi Summary in which a person's sleep is analyzed by a myriad of different on-body sensors (e.g., EEG, EMG, ECG).PSG is considered the gold standard for sleep screening, yet, the PSG measurements are uncomfortable, disturb the natural sleeping behavior and therefore lack reproducibility, and require often time consuming manual analysis.To alleviate these shortcomings, we investigate a new, fully automated, and less invasive monitoring approach.We focus on a video camera-based system for sleep analysis, consisting of a near infrared (NIR) camera and NIR light source.We present methods to extract activity levels, sleep efficiency scores, breathing information, body part movements, infant sleeping pose, and wake-up behavior.Challenging conditions such as a shared bed environment, different camera locations and moving cast shadows are taken into account.We designed a contactless, off-body video actigraphy system to monitor a sleeping subject's movements.With the aim to analyze competitiveness with wrist actigraphy, we conducted a differentiated comparison between the two actigraphy methods.Video actigraphy contains more comprehensive information and is generally more sensitive than wrist actigraphy.The average PSG to video based sleep efficiency error is comparable to the PSG to wrist actigraphy based error.In order to discriminate movements of different body parts, we investigated an enhanced K-Means clustering approach for motion vectors.When performing ME on sleep sequences, large environmental variations between recording situations such as viewing angles, blanket types, zoom factors and illumination conditions, can yield different motion vector fields for similar movements.Therefore, our multi-distance clustering algorithm is computing content-dependent weights and is not only based on spatial distances between data points but also on motion vector angle and length.To realize an easy-to-install system for the end user, we investigated an installation where the camera is conveniently placed on the bedside table of the primary subject who is to be monitored.We designed a method sensitive to small scale movements so that not only activity levels are monitored but also the respiratory waveform can be computed.Our breathing analysis method performed admirably with an overall sensitivity of 87%, precision of 90%, and a breathing rate correspondence of 93%, surpassing the results of state-of-the-art video based breathing algorithms.To discriminate small movements of a subject from moving cast shadows on a non-planar and dynamic background, our video processing method integrates motion detection, motion estimation and texture analysis, efficiently aggregated in a strong classifier using cascaded AdaBoost.Movement event classification improved threefold with the proposed method in highly varying lighting conditions, compared to stateof-the-art.The first lifestyle application we investigated illustrates an intelligent baby monitor that warns parents when their baby is turning in its sleep to its belly.By designing a turning movement detector and combining its information with face detection results, we improved the infant's sleeping pose accuracy by 11% compared to a method using xii solely face detection.A personalized wake-up system is envisioned in the second lifestyle application that exposes the sleeping subject to light that is adapting its intensity over time according to the subject's measured activity level.Therefore, we developed a system which can measure the sleeping person's activity and control the light output such that the subject's behavior corresponds to an activity trajectory of a favorable wake-up experience.xiii