The steps to create face images include generating geometric face shapes and rendering face appearance with 4 skin tones and 21 roughness levels. To render images, a physically based ray-tracing algorithm was used. The algorithm simulated a camera to generate an image film. From a point of the image film, a generated ray traveled and intersected with a surface. To obtain the color values at each image point, the surface normal, the geometric and radiometric information of the illuminant, and material optical properties were the inputs of the rendering engine. The geometric shape of the face providing surface normals was the initial model of FaceBuilder [27], which is a gender-neutral 3D face model without textures and shadings. To illuminate the environment, light-map rendering was utilized with a high dynamic range image shown in Figure 1. The light map had outdoor natural illumination and buildings in the scene. Thus, it simulates natural illumination in the real world, avoiding excessively high luminance that causes overexposure and overly low luminance that leads to a lack of contrast. The buildings in the scene introduced some texture into the images, which provided visual cues for gloss judgment [56, 61]. The material optical properties were also defined by a bidirectional scattering surface reflectance distribution function (BSSRDF). In this study, the skin BSSRDF was estimated with a given diffuse scattering coefficient, transmission coefficient (set at 1), microfacet roughness, refractive index (set at 1.55), and mean free path ([0.0013, 0.0009, 0.0006] for RGB channels). The mean free path, the mean distance of light path in a medium before scattering, is the reciprocal of attenuation coefficient, ranging from 0 to 1. As shown in Figure 2 and Table A.1, 21 levels of roughness were used to generate the experimental stimuli. The selected physically based rendering engine used the Beckmann–Spizzichino model as the microfacet distribution reflection function, and the roughness in the model represents the variation in slopes of microfacets in a unit area [41]. To represent the diffuse optical property of the material, the engine approximated spectral reflection with three-channel RGB values [41, 45]. Other than the material optical parameters, the camera and environmental parameters were kept unchanged.

Using the ray-tracing technique, four different skin type face models were generated. For each skin type, 21 face objects with varying roughness levels were created. The four skin tones were selected through clustering the Pantone SkinTone Guide [40] color sets (measured with an i1Pro spectrophotometer) by using the k-means clustering algorithm. We converted the reflectance of the skin set into a perceptual color space (CAM16 UCS) for clustering. From the resulting eight cluster centers, four representative and distinct colors were chosen. Then, the Jab of CAM16 UCS was converted into sRGB as the input of rendering (Figure 4). The rendered face models at various roughness levels and selected skin colors are shown in Figures 3 and 5, respectively. It should be noted that the colors in Fig. 5 are from the final renderings. The colors selected from the clustering results and used as the diffuse RGB parameters do not exactly match the final renderings, as the rendering process involves additional parameters beyond diffuse characteristics. The final stimulus colors in Fig. 5 are labeled “lightest,” “light,” “dark,” and “darkest” to indicate relative lightness differences but do not correspond to any specific skin typology (e.g., Fitzpatrick, Monk).

Eleven observers participated in this experiment (all had normal or correct-to-normal visual acuity, and normal color vision; 4 males and 7 females of age from 20 to 35). Observers were presented with one skin color stimulus set at a time and asked to arrange the 21 roughness level images along a horizontal scale by dragging and dropping them with their mouse, aligning them with their perceptions of gloss relative to the left and right anchor images at the bottom of the experimental interface (see Figure 6). Observers were instructed that the relative distance between the images along the horizontal direction should be linear with respect to their perceived gloss (e.g., images could overlap if they were perceptually close, and images could have unequal spacing between them). The positions of the images along the vertical direction were not recorded, but observers could use the vertical dimension to avoid image overlap, which could be useful for comparing images simultaneously. The images used for left and right anchors were chosen to be those having minimum and maximum roughness values, respectively. The images are resized so that the field of view of each image subtends 12∘ in width and 16∘ in height, 320 × 440 in pixel width and height. An example trial is shown in Fig. 6.

Corresponding to the four skin color types, each observer performed four trials. In each trial, 21 images with various roughness levels were shown simultaneously. The starting position of the images before being moved to the scale were displayed in random order. Observers arranged each of the 21 images before moving on to the next trial. The order of trials for a given skin color was also randomized for different observers.

The spatial coordinates of each image placed by observers along the horizontal axis in the experimental interface were recorded and converted to gloss scales. The left anchor image representing the most glossy was assigned a value of 1, and the right anchor image representing the most matte was assigned a value of 0. The collected perceived gloss was used in this experiment as the dependent variable. Given the large number of images, two monitors were used (side by side) for the experiment. Both monitors were calibrated to the sRGB color profile with D65 white points (luminance: 100 cd/m2) with primaries R (0.64, 0.33), G (0.3, 0.6), and B (0.15, 0.06) in CIExy. Observers completed the experiment under a dim lighting condition, and their heads were at a fixed position facing the center of two monitors. Anti-glare monitors were used with the glare shield on. Observers were not able to see highlights and reflections from monitor surfaces under the dim lighting.

The experiment contains 84 stimuli. On the left side of the experimental interface, an object image would appear with a randomly selected shape (face, sphere, or blob), baseline color type (“lightest,” “light,” “dark,” “darkest”), and gloss level (seven total levels). The face-shaped objects were the same as those used in Experiment 1. We additionally chose to include the sphere-shaped object to represent a simple geometric shape with smooth, predictable material and light interactions. The blob-shaped object was chosen to represent a more complex shape with less predictable light interaction (edges and contours similar to a face) but with less familiarity and social relevance than a face. The seven gloss levels of the objects were determined by the roughness function established in Experiment 1, so the roughness values used in Experiment 2 increase approximately linearly with perceived gloss (Figs. 7, 8, and 11). The gloss levels used represent ranges of perceived gloss from 0.2 to 0.7 with an interval of 0.08. The four baseline color types were the same as those used as in Experiment 1.

Fifteen observers participated in the experiment (age from 25 to 50; 9 females and 6 males). They were familiar with color space dimensions. All had normal or correct to normal visual acuity, and normal color vision. Eleven of them had previously participated in the first experiment. One object target image was displayed on the left side of the screen for each trial. This target image had a randomly selected shape type, baseline color type, and gloss level. The color patch was displayed on the right side of the screen simultaneously. Observers were asked to adjust the color of the patch to match the target’s surface color. Observers were able to adjust the color patches by using a keyboard along the dimensions of lightness (CIELAB L*, using up/down keys) and chroma (CIELAB C*, using left/right keys). The base colors of the patch were those extracted from a point sample of the forehead region of the rendered face images with 0 roughness. These point samples were selected as they did not include either specular highlights or shadows, and they were judged by the authors as having the most diffuse area of the images. The full adjustment range was ±20 L* and ±10 C* of the base color of patches. The initial color was randomized. The color path could be adjusted to either increase or decrease along the L* and C* dimensions independently, and the step size of each adjustment (key press) was 1 unit for each dimension (Figs. 9 and 10). When observers were satisfied with their color match, their response was recorded. The responses included the final CIELAB L*, C*, a*, and b* values of the color patch as well as the L* and C* indices (representing the steps of L* and C* adjustment, respectively). Each observer completed a trial for each combination of target shape type (3), baseline color type (4), and gloss level (7) for a total of 84 trials.

There was a significant main effect of gloss level on lightness adjustment (F (1,14) = 14.54, p = 0.002), indicating that perceived lightness generally decreased as gloss increased (B = −0.91, SE = 0.24). The main effect of shape type did not reach statistical significance (F (2,13) = 3.21, p = 0.074). However, followup t-tests indicated that faces (M = 32.1, SE = 0.83) were generally perceived as darker than blobs (M = 34.2, SE = 0.66; t (14) = 3.46, p = 0.004) but not spheres (M = 32.4, SE = 0.81; t (14) = 0.42, p = 0.68). There was a significant main effect of color type on lightness adjustments (F (3,12) = 10.07, p = 0.001). Generally, participants increased lightness more for “lightest” colors (M = 35.2, SE = 0.65), followed by “light” (M = 33.6, SE = 0.66), “dark” (M = 33.2, SE = 0.70), and “darkest” (M = 29.5, SE = 1.03) color types. All pairwise differences among the color types were significant (p < 0.005) except for the difference between “light” and “dark” (t (14) = 1.21, p = 0.248); see Figures 12(a) and 13.

There was also a significant gloss level and shape type interaction (F (2,13) = 4.052, p = 0.043), indicating that the influence of gloss on lightness varied as a function of shape type. Perceived lightness decreased as gloss increased for face-shaped objects (B = −0.55, SE = 0.13; t (14) = 4.29, p < 0.001), but this pattern was not statistically significant for spheres (B = −0.13, SE = 0.12; t (14) = 1.12, p = 0.28) or blobs (B = −0.15, SE = 0.12; t (14) = 1.21, p = 0.25); see Figures 13 and 14.

There were no statistically significant interactions between gloss level and color type (F (3,12) = 1.27, p = 0.33), between shape type and color type (F (6,9) = 1.32, p = 0.34), or the three-way interaction among them (F (6,9) = 0.74, p = 0.63); see Fig. 14.

Our analysis also shows that there is no significant effect of gloss level (F (1,14) = 0.01, p = 0.92) and shape type (F (2,13) = 1.32, p = 0.30) on chroma adjustments. But a significant main effect on chroma (F (3,12) = 3.56, p = 0.047) due to color type is observed. Generally, perceived chroma decreased from “lightest” (M = 12.0, SE = 0.31) to “light” (M = 11.6, SE = 0.22), “dark” (M = 11.5, SE = 0.21), and “darkest” (M = 10.7, SE = 0.14) color types. All pairwise differences among the color types were significant (p < 0.013) except for the difference between “light” and “dark” (t (14) = 1.10, p = 0.288); see Fig. 12(b). Moreover, no statistically significant two-way or three-way interactions were observed among these variables (p > 0.41); see Figures 15, 12(b), and 16).

In addition to gloss level analysis on color appearance, we explored whether observer color adjustments correspond to systematic patterns among regions of the images themselves. To do this, we identified the image pixels that comprised CIELAB values having small color differences (ΔE ≤ 5) from the average CIELAB values of observers’ responses. These pixels are shown in color with pixels outside this range colored gray in Figure 17. For each shape at each gloss level, ΔE is computed and averaged across the four color types such that the images shown identify the objects’ regions of interest rather than the precise color of each object. This analysis speculates about the strategy observers might have used to determine their representative color matches. However, it is worth noting that their decisions may not have been determined according to specific areas of the objects necessarily. Additional data such as from eye-tracking or pixel-selection methods in future work may be needed to confirm these strategies. Because perceived chroma was not impacted by changes in gloss, this section focuses on discussions regarding perceptions of object lightness. These analyses indicate that the image areas containing similar colors as observer responses are largely near the edges surrounding the specular highlight, suggesting that observers tend to use the brightest region of the objects (but excluding the highlight area) when judging colors of the surfaces. This strategy likely becomes easier as objects become more glossy as the specular highlight edge becomes more defined. This possibility is likely supported by the general decrease in lightness adjustments as gloss increased, as highlight regions are more likely to be avoided and considered distinct from perceptions of the surface color. The colored area is larger when gloss level decreases for spheres and blobs but not faces (Fig. 17). This indicates that the area representing surface colors by observers is larger with decreasing gloss, possibly because the boundary between diffuse reflection and specular highlight is less sharp for more matte surfaces. It has been previously shown that perceived lightness of objects is determined by diffuse reflection instead of mean lightness of objects [14, 57] and that observers judge the lightness of an object according to the area neighboring the highlight region [14], which is supported by the current work.