The human face is an essential stimulus in our social life and occupies a large proportion of digital content. The perceptual appearance of faces is important in computer vision, psychology, digital media, and related fields. Various structural and color features of faces have been studied over the years. However, the study of perceptual glossiness of faces and its influence on facial skin color appearance is very limited. This study investigates the relationship between perceived glossiness of facial skin, skin roughness, and perceptual color attributes. Psychophysical studies were conducted to model perceptual gloss and its effects on the perceived color appearance of faces. The investigation was carried out across varying roughness levels and skin tones. The results indicated that facial gloss influenced the perceived lightness of the facial skin, a phenomenon not observed to the same extent in non-face objects included in the experiment. The effect on lightness could partially be explained by a strategy of discounting specular components for surface color perception. Observers tended to focus on the brightest regions of the objects while avoiding specular highlights to infer color attributes. The current findings provide insights into understanding visual appearance characteristics of face and non-face objects and will be useful for accurate gloss and color reproduction of graphical generated faces.