Back to articles
Articles
Volume: 31 | Article ID: art00013
Image
Semantic label bias in subjective video quality evaluation: a standardization perspective
  DOI :  10.2352/ISSN.2470-1173.2019.10.IQSP-311  Published OnlineJanuary 2019
Abstract

Recent studies brought to light that the semantic labels (e.g. Excellent, Good, Fair, Poor, and Bad) commonly associated with discrete scale ITU subjective quality evaluation induce a bias in MOS computation and that such a bias can be quantified by some reference coefficients which are independent with respect to the observers panel. The present paper reconsiders these results from a standard upgrading perspective. First, it theoretically investigates the way in which results obtained on semantically labeled scales can be “cleaned” from such an influence and derives the underlying computation formula for the mean opinion score. Secondly, it suggests a unitary evaluation procedure featuring both semantic free MOS computation and backward compatibility with respect to state-of-the-art solutions. The theoretical and methodological results are supported by subjective experiments corresponding to a total of 440 human observers, alternatively scoring 2D and stereoscopic video content. For each type of content, both high and low quality excerpts are alternatively considered. For each type of content and for each type of quality a 5 level (Excellent, Good, Fair, Poor, and Bad) grading scales is considered.

Subject Areas :
Views 31
Downloads 0
 articleview.views 31
 articleview.downloads 0
  Cite this article 

Mihai MITREA, Rania Bensaied, Patrick Le Callet, "Semantic label bias in subjective video quality evaluation: a standardization perspectivein Proc. IS&T Int’l. Symp. on Electronic Imaging: Image Quality and System Performance XVI,  2019,  pp 311-1 - 311-8,  https://doi.org/10.2352/ISSN.2470-1173.2019.10.IQSP-311

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2019
72010604
Electronic Imaging
2470-1173
Society for Imaging Science and Technology
7003 Kilworth Lane, Springfield, VA 22151 USA