Back to articles
Articles
Volume: 33 | Article ID: art00020
Image
A Comprehensive Analysis of Crowdsourcing for Subjective Evaluation of Tone Mapping Operators
  DOI :  10.2352/ISSN.2470-1173.2021.9.IQSP-262  Published OnlineJanuary 2021
Abstract

Tone mapping operators (TMO) are pivotal in rendering High Dynamic Range (HDR) content on limited dynamic range media. Analysing the quality of tone mapped images depends on several objective factors and a combination of several subjective factors like aesthetics, fidelity etc. Objective Image quality assessment (IQA) metrics are often used to evaluate TMO quality but they do not always reflect the ground truth. A robust alternative to objective IQA metrics is subjective quality assessment. Although, subjective experiments provide accurate results, they can be time-consuming and expensive to conduct. Over the last decade, crowdsourcing experiments have become more popular for collecting large amount of data within a shorter period of time for a lesser cost. Although they provide more data requiring less resources, lack of controlled environment for the experiment results in noisy data. In this work1, we propose a comprehensive analysis of crowdsourcing experiments with two different groups of participants. Our contributions include a comparative study and a collection of methods to detect unreliable participants in crowdsourcing experiments in a TMO quality evaluation scenario. These methods can be utilized by the scientific community to increase the reliability of the gathered data.

Subject Areas :
Views 30
Downloads 3
 articleview.views 30
 articleview.downloads 3
  Cite this article 

Ali Ak, Abhishek Goswami, Patrick Le Callet, Frédéric Dufaux, "A Comprehensive Analysis of Crowdsourcing for Subjective Evaluation of Tone Mapping Operatorsin Proc. IS&T Int’l. Symp. on Electronic Imaging: Image Quality and System Performance XVIII,  2021,  pp 262-1 - 262-7,  https://doi.org/10.2352/ISSN.2470-1173.2021.9.IQSP-262

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2021
72010604
Electronic Imaging
2470-1173
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane Springfield, VA 22151 USA