In area of white balance, the process of large colored background images seems to be a problem. Regarding this issue, a white balance algorithm based on facial skin color was proposed. A neural network-based object detection algorithm and an adaptive threshold segmentation algorithm were combined to achieve the accurate segmentation of skin color pixels. Then a 3-dimension color gamut mapping method in CIELAB color space was used to do the illumination estimation. Last, CAT16 model was applied to rendering the images to standard lighting condition. Besides, an ill white balanced images dataset taken against large colored backgrounds were prepared to test the present algorithm and others’ performance. The results show the proposed algorithm performs better on the dataset.