Reliability of Objective Picture Quality Measures
Sonja Grgi, Mislav Grgi, Marta Mrak · 2004
This paper investigates a set of objective picture quality measures for application in still image compression systems and emphasizes the correlation of these measures with subjective picture quality measures. Picture quality is measured using nine dieren t objective picture quality measures and subjectively using Mean Opinion Score (MOS ) as measure of perceived picture quality. The correlation between each objective measure and MOS is found. The eects of dieren t image compression algorithms, image contents and compression ratios are assessed. Our results show that some objective measures correlate well with the perceived picture quality for a given compression algorithm but they are not reliable for an evaluation across dieren t algorithms. So, we compared objective picture quality measures across dieren t algorithms and we found measures, which serve well in all tested image compression systems. K e y w o r d s: correlation, JPEG, JPEG2000, objective assessment, picture quality measures, SPIHT With the increasing use of multimedia technologies, image compression requires higher performance. To address needs and requirements of multimedia and Internet applications, many ecien t image compression techniques, with considerably dieren t features, have recently been developed. Image compression techniques exploit a common characteristic of most images that the neighboring picture elements (pixels, pels) are highly correlated [1]. It means that a typical still image contains a large amount of spatial redundancy in plain areas where adjacent pixels have almost the same values. In addition, still image can contain subjective redundancy, which is determined by properties of human visual system (HVS). HVS presents some tolerance to distortion depending upon the image content and viewing conditions. Consequently, pixels must not always be reproduced exactly as originated and HVS will not detect the dierence between original image and reproduced image [2]. The redundancy (both statistical and subjective) can be removed to achieve compression of the image data. The basic measures for the performance of a compression system are picture quality and compression ratio (dened as ratio between original data size and compressed data size). In lossy compression scheme, image compression algorithm should achieve trade o between compression ratio and picture quality. Higher compression ratios will produce lower picture quality and vice versa. The evaluation of lossless image compression techniques is a simple task where compression ratio and execution time are employed as standard criteria. The picture quality before and after compression is unchanged. Contrary, the evaluation of lossy techniques is dicult task because of inherent drawbacks associated with both objective and subjective measures of picture quality. Objective measures of picture quality do not correlate well with subjective quality measures [3], [4]. Subjective assessment of picture quality is time consuming process and results of measurements should be processed very carefully. In many applications (photos, medical images where loss is tolerated, network applications, World Wide Web, etc.) it is very important to choose image compression system which gives the best subjective quality, but the quality has to be evaluated objectively. Therefore, it is important to use objective picture quality measure, which has high correlation with subjective picture quality. In this paper we attempt to evaluate and compare objective and subjective picture quality measures. As test images we used images with dieren t spatial and frequency characteristics. Images are coded using JPEG, JPEG2000 and SPIHT compression algorithms. The paper is structured as follows. In section 2 we dene picture quality measures. In section 3 we briey present image compression systems used in our experiment. In Section 4 we evaluate statistical and frequency properties of test images. Section 5 contains numerical results of picture quality measures. In this section we analyze correlation of objective measures with subjective grades and we propose objective measures, which should be used in relation to each image compression system, and objective measures, which are suitable for the comparison of picture quality between dieren t compression systems.