Validation and Evaluation for Visual Attention Models
Liming Zhang, Weisi Lin · 2013
This chapter assesses the performance of the saliency detection models described in the previous chapters. As with many other cases in engineering, a developed visual attention model needs to be critically benchmarked against other models, and then fully tested before being used in particular applications and situations. A number of qualitative and quantitative evaluation methods, as well as related ground-truth databases, are introduced in this chapter. Common benchmarks include simple man-made visual patterns, human-labelled images and eye tracking data, which are first given in Sections 6.1–6.3. The quantifying estimation of performance of the computational models is listed in Sections 6.4–6.6. The most commonly used criteria are PPV, TPR, F-measure, ROC and AUC, as introduced in Section 6.4. The statistical criteria for both static and dynamic scene – NNS and KL distance – are presented in Section 6.5. Then Section 6.6 shows the criterion of Spearman's rank-order correlation with visual conspicuity. Each type of ground-truth, the associated evaluation methods and their advantages and disadvantages are discussed whenever needed and possible.