In this paper we focus on the problem of image segmentation by color clustering. We present a robust agglomerating clustering algorithm based on cluster validity criteria derived from fuzzy partitions. The result is a simplified segmentation having a small number of large regions. The interest of the proposed method is that it requires a single parameter and that the computational complexity is very low.