Subjective quality assessment for stereoscopic video: Case study on robust watermarking

This paper investigates three key issues related to full reference subjective quality evaluation tests for stereoscopic video, namely, the number of quality levels on the grading scale, the number of observers in the evaluation panel, and the inter-gender variability. It is theoretically demonstrated that the scores assigned by the observers on a continuous grading scale can be a posteriori mapped to any discrete grading scale, with controlled statistical accuracy. The experiments, performed in laboratory conditions, consider image quality, depth perception and visual comfort. The original content (i.e. the full reference) is represented by the 3DLive corpus, composed of 2 hours 11 minutes of HD 3DTV content. The modified content (i.e. the content to be evaluated) is obtained by watermarking this corpus with four methods. A panel of 60 observers (32 males and 28 females) was established from which further randomly selected sub-panels of 30 and 15 observers were also subsequently extracted. In order to simulate a continuous scale, the subjective evaluation was carried out on 100 quality levels, which are a posteriori mapped to discrete scales of q quality levels, with q between 2 and 9. The statistical investigation focused on the Mean Opinion Score and considered three types of statistical inferences: outliers detection, confidence limits, and paired t-tests.