A vision towards automatic inference of hostile intent from sensory observations
Gertjan J. Burghouts, Klamer Schutte · TNO Repository · 2011
Amsterdam, The Netherlands, 27th of May 2010. The national security authorities call out tosecure the area around the central train station. 150 Security officers, both military and police, areinstructed to guard the area and to pick out those people with hostile intent. Although the officerswere very effective in assessing people’s nervousness and where they were looking, - a lot of otherindicators were hard for them to recognize. In many cases the officers did not recognize particularindicators. At TNO, we have a vision on the technology to support them. So how to solve thepuzzle of discovering somebody with bad intentions?Evidence exists that hostile intent can be detected from particular behaviors. Psychologicalexperiments with CCTV videos demonstrated that humans can very well assess whether particularbehaviors hint at a future escalation. Also, from analysis of other attacks, we have learned thatsome clues were visible from the surveillance cameras. This observation led to a compilation ofso-called ‘deviant behaviors’. The list is categorized by indicators that relate to respectively theappearance of a person, his or her specific behavior and objects that are carried, touched orexchanged.With these ‘deviant behaviors’, the problem of ‘intent recognition’ can be decomposed intorealistic challenges of early detection of specific behavioral indicators. We distinguish betweenfour types of features: (1) trajectories (e.g. where did somebody move, with whom did the personinteract), (2) appearance of the person as a whole (e.g. the pose and entropy), (3) body parts andspatiotemporal features, (4) physiological properties (e.g. the change of body temperature, sweator heart beat). For category 4 we investigated whether vital life signs can be monitored at adistance, together with a renowned health-care innovator. Whereas this is still very rudimentary,current research focuses on categories 1, 2 and 3. These properties demonstrated to be verydiscriminative, especially when they are combined. Inspired by trained police and soldiers, wehave build software to combine behavioral indicators on the spot, the ‘0+0+0=1’ principle.