We explore the influence of surface and subsurface reflections on skin gloss perception. We rely on multimodal photography to separate the surface and subsurface reflection images. Since the original data consists of a limited number of images (25 subjects, front and side view, before and after skin cleansing), we apply different transformations to surface and subsurface reflection images, in order to generate a broad range of appearance of skin images. We conducted two empirical studies with the expanded set of data, at both the macro-scale level (whole face) and the meso-scale level (local skin patch). We found that increasing the contrast of surface reflection results in higher gloss perception, while a decrease in the amount of subsurface reflection (lower average lightness, darker complexion) results in higher gloss perception; however, the differential effect of subsurface reflection on gloss diminishes as the average lightness becomes very low. We also computed the statistics of the two reflection images and found their effects (sometimes opposite for the corresponding statistic) on gloss perception. We then learned a regression model based on the concatenation of statistics from the surface and subsurface reflection images to predict relative gloss differences. Our results indicate that using the statistics from both modalities provides more consistent correlation with human judgments than using only the statistics from a single modality.