In this paper, we study individual quality scores given by different observers for various image distortions (saturation, contrast, and color quantization) at different levels. We created a database that contains a total of 232 images, derived from 21 pristine images, three distortions, and five levels. The database was rated by 31 participants collected through an online platform. The study shows that observers have distinguishable patterns with respect to different distortions. Using quadratic regression models, we visualized the behavior patterns of different groups of observers. The database and the individual scores collected are publicly available and can be further used for quality assessment research.
In recent years, 3D reconstruction systems comprising multiple depth sensors have received increasing interest for dynamic scene reconstruction and related applications. Publicly available ground truth data are of limited usefulness when dealing with quality assessment of self-recorded data delivered by customized stereo configurations. In this paper, we propose a framework that incorporates versatile strategies for quantitative and qualitative evaluation of a multi-stereo reconstruction system and its intermediate products. Besides the design of suitable calibration objects for quantitative measurements, the framework exploits multiview data redundancy and generated novel views for objective quality assessment and to obtain subjective ratings from users. We demonstrate the applicability of our evaluation system in experiments with several stereo matching algorithms and view fusion approaches along with a pair-comparison based user study. We believe that our proposed evaluation framework is beneficial for the assessment of 3D products derived from self-recorded dynamic data of comparable set-ups, for example, in the context of subsequent augmented reality applications.