We describe a method for finding the eyes and mouth in a color image of a human face. This method makes use of two new models: a skin color model and a three-dimensional (3-D) geometric model of facial features. The skin color model is based on a medical model for the reflectance of skin and applies to all kinds of skin color. The 3-D geometry model is based on medical statistics of dimensions of the human face and is used to evaluate the geometry of facial features. This model applies for any orientation of the face. The algorithm first determines candidates of facial features by color segmentation. Sets of mouth candidates and eye candidates are evaluated based on the color of the segment that surrounds them and its geometry using the skin color model and the 3-D geometry model. The features are recognized according to the candidate set with the highest likelihood. In experiments, the algorithm detects the features correctly in a large majority of face images in many orientations with a cluttered background.