Determination of the role of observer experience is of potentially critical concern regarding the development of accurate color difference formulae. As part of a larger multivariable experiment investigating the minimum inter- and intra-observer variability possible among a statistically significant set of observers, a pilot study has been conducted to compare the performance of 25 naïve vs. 25 expert visual assessors for a set of 31 pairs of colored textile samples using a controlled psychophysical grayscale method.No evidence of a training effect among the naïve observers was found using this method following three repeat assessments by each observer. However, a statistically significant difference between the judgments made by naïve and expert observers was found, demonstrating that observer experience is an important consideration in the development of visual datasets. The intra-observer variability among the naïve observers was equivalent to that of the expert assessors.
Lina Cárdenas, David Hinks, Renzo Shamey, Rolf Kuehni, Warren Jasper, Melih Gunay, "Comparison of Naïve and Expert Observers in the Assessment of Small Color Differences between Textile Samples" in Proc. IS&T CGIV 2006 3rd European Conf. on Colour in Graphics, Imaging, and Vision, 2006, pp 341 - 344, https://doi.org/10.2352/CGIV.2006.3.1.art00070