On the Properties of Mean Opinion Scores for Quality of Experience Management
Jie Xu, Liyuan Xing, Andrew Perkis, Yuming Jiang · 2011
For research on quality of experience (QoE), mean opinion scores (MOS) are widely chosen as the results of subjective tests and the ground-truth reference for further research on objective quality modeling. Furthermore, the results of objective quality modeling are used for QoE management subsequently. Therefore, the performance of QoE management process actually depends heavily on MOS. However, the rationality of MOS for QoE management is not yet technically proven in the literature. In this paper, we first prove that subject homogeneity is implicitly assumed for obtaining MOS by modeling the arithmetic averaging process from a systematic viewpoint. However, we point out that actually subjects exhibit variability in terms of quality assessment. Then we elaborate that this mismatch may results in failures if we conduct QoE management based on MOS. Finally we propose a utility-based averaging method (uMOS) which improves the performance of QoE management.