Mining rare and frequent events in multi-camera surveillance video using self-organizing maps
Valery A. Petrushin · 2005
This paper describes a method for unsupervised classification of events in multi-camera indoors surveillance video. This research is a part of the Multiple Sensor Indoor Surveillance (MSIS) project which uses 32 AXIS-2100 webcams that observe an office environment. The research was inspired by the following practical problem: how automatically classify and visualize a 24 hour long video captured by 32 cameras? Raw data are sequences of JPEG images captured by webcams at the rate 2-6 Hz. The following features are extracted from the image data: foreground pixels' spatial distribution and color histogram. The data are integrated by event by averaging motion and color features and creating a "summary" frame which accumulates all foreground pixels of frames of the event into one image. The self-organizing map (SOM) approach is applied to event data for clustering and visualization. One-level and two-level SOM clustering are used. A tool for browsing results allows exploring units of the SOM maps at different levels of hierarchy, clusters of units and distances between units in 3D space. A special technique has been developed to visualize rare events. The results are presented and discussed.