Extending subjective experiments for image quality assessment with baseline adjustments
Ping Zhao, Marius Pedersen · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2015
In a typical working cycle of image quality assessment, it is common to have a number of human observers to give perceptual ratings on multiple levels of distortions of selected test images. If additional distortions need to be introduced into the experiment, the entire subjective experiment must be performed over again in order to incorporate the additional distortions. However, this would usually consume considerable more time and resources. Baseline adjustment is one method to extend an experiment with additional distortions without having to do a full experiment, reducing both the time and resources needed. In this paper, we conduct a study to verify and evaluate the baseline adjustment method regarding extending an existing subjective experimental session to another. Our experimental results suggest that the baseline adjustment method can be effective. We identify the optimal distortion levels to be included in the baselines should be the ones of which the stimulus combinations produce the minimum standard deviations in the mean adjusted Z-scores over all human observers in the existing rating session. We also demonstrate that it is possible to reduce the number of baseline stimuli, so the cost of extending subjective experiments can be optimized. Comparing to conventional researches mainly focusing on case studies of hypothetical data sets, we perform this research based on the real perceptual ratings collected from an existing subjective experiment.