Parametric Evaluation of Video Motion Tracking Data Sets
Mukesh C. Motwani, Rakhi C. Motwani · 2010
Video tracking is a complex problem because the environment, in which video motion needs to be tracked, is widely varied based on the application and poses several constraints on the design and performance of the tracking system. Current datasets that are used to evaluate and compare video motion tracking algorithms use a cumulative performance measure without thoroughly analyzing the effect of these different constraints imposed by the environment. But it needs to analyze these constraints as parameters. The objective of this paper is to identify these parameters and define quantitative measures for these parameters to compare video datasets for motion tracking. dataset used at the call for real-time event detection solutions (CREDS) used to test the performance in indoor environments. There are no publicly available video datasets for applications such as monitoring movement of ships in dockyard. The current evaluation systems compare algorithms in very specific environments against a single metric related to deviation of the tracked path from the actual ground truth. This metric is cumulative and does not identify the cause of deviation or measure the deviation due to specific outlier. This can result in the failure of the evaluated algorithm in an environment with certain unaccounted conditions such as occlusion since there is no object in the scene and thus absence of ground truth. The current evaluation metrics based on track (path of object of ground truth is compared with path of object of tracker) or frame based evaluations (4),(5) are not sufficient to cover all the operating scenarios for tracking algorithms. Thus, no generic evaluation metrics exist which can be used to test the performance of tracking algorithms in the presence of outliers. This lack of analysis results in a flawed method of comparison which leads to poor selection of tracking algorithms for a system. It is a challenge to arrive at a true comparison metric for tracking systems. The metric of deviation from ground truth path can still be used and is a valid measure provided image sequences account for these variations in isolation. Thus, there is a need to create a cumulative metric which is derived from these subjective metrics corresponding to different constraints in the environment. It is imperative to identify these constraints in the environment for video tracking algorithms to be benchmarked. There is also a need to create datasets that