Foveate wavelet transform and its applications in digital video processing, acquisition, and indexing
Ze-Nian Li, Jie Wei · 1998
A new Variable Resolution (VR) representation—Foveate Wavelet Transform (FWT) is proposed to represent video frames in an effort to facilitate efficient and effective visual content representation and extraction by simulating animate vision systems. Compared to other existing VR techniques, the benefits of the proposed representation encompass direct processing and analysis within the transform domain, orientation selectivity, and flexibility in emulating animate vision systems. Techniques with regard to the digital video processing, acquisition, and indexing based on the FWT representation are developed. Video processing is treated as a low-level image preprocessing step for future information retrieval from videos. Based on the FWT, two techniques are developed: (a) Motion compensation: we developed the Enhanced Multiple Resolution Motion Compensation, where the time-consuming motion estimation process is only conducted in the blocks containing the potential moving area. (b) Motion field estimation: a Maximum a Posteriori - Markov Random Field (MAP-MRF) method based on Mean Field Theory is developed to obtain a more refined motion field which can better reflect the video's visual contents. The FWT representation is also exploited for active acquisition of videos. The active single camera control is first developed, we then proceed to consider effective stereo camera control by incorporating depth information. With these approaches, cameras can move actively in the video acquisition process in response to the object motion based on the FWT. Finally, the subject of video indexing is studied in which two important issues are addressed: (a) Camera motion recovery: this is the reverse of the problem of active camera control, i.e., recovering the camera motion from a video clip taken by human operators. Camera motions, such as pan/tilt, zoom, and combinations of them, are computed based on the behavior of the dense motion vectors estimated by the MRF-based method, whereby the efficacy of the FWT is further delineated. (b) Object-based video indexing: a preliminary object motion-based video indexing method is developed where the visual contents including an object of central interest and attention shifts in the FWT-based active video are indexed, so that subsequent object-based queries can be conducted effectively.