Facial diagnosis is an important diagnostic method in Japanese-traditional (Kampo) medicine. Major disease states by facial diagnosis are blood stagnation, blood deficiency, and yin deficiency. These facial diagnoses are subjective and empirically obtained. To solve these problems, we proposed to construct a system to output the score (1 to 5) for evaluating pathophysiology of the patient by using facial image obtained by RGB camera. We evaluate this system by calculating mean squared error (MSE) between the score given by medical doctor and estimated by the system. Our method achieved to estimate the score accurately as the MSE is less than 1.0. From the results of construction of the system, we found the important regions of the face for diagnosing disease states by medical doctor using the method of significant feature selection.