On the Opinion Score Consistency in Repeated 360° Video Quality Assessment for Standing and Seated Viewing on Head-Mounted Displays
Majed Elwardy, Hans-Jurgen Zepetnick, Thi My Chinh Chu, Yan Qing Hu · 2021
The development of immersive media systems and services relies on subjective tests that provide ground truth on the quality of experience as perceived by humans. In this paper, we investigate the opinion score consistency through a repeated subjective 360° video quality assessment experiment. The test stimuli are presented on a head-mounted display with participants given the task to rate the video quality in three repeated sessions for both standing and seated viewing. The statistical analysis of the data from the subjective tests aims at revealing if participants change their rating behavior over time or keep their opinion scores given to the test stimuli in standing and seated viewing consistent in a statistical sense. The experimental results are reported in terms of histograms of opinion scores, skewness and kurtosis of opinion scores, mean opinion scores, standard deviation of opinion scores (SOS), SOS fitting functions, and analysis of variance tests. The statistical analysis supports the conjecture that each participant has its own but consistent rating behavior throughout the respective sessions for standing and seated viewing. The finding that the quality rating behavior of the individual participant does not fundamentally change over time may, e.g., assist in scheduling subjective tests under pandemic conditions where experimental campaigns may need to be stalled for an unknown period of time.