We studied the utilization of illumination and observer spectral characteristics in clustering based color image segmentation. Segmentation was based on k-means algorithm, spectral clustering algorithm and non-local spatial constraint spectral clustering algorithm both with the Nyström method. The image segmentations were performed for four different representations derived from a set of spectral images: 1) simulated sRGB images using D65 illumination, 2) six band images based on sRGB images under the D65 and A illuminations, 3) estimated spectral reflectance images, and 4) true spectral reflectance images. The spectral reflectance estimation was based on Wiener estimation model, reflectance of Macbeth Color Checker (24 samples) and CIE XYZ tristimulus values under the D65 and A illuminations. The experimental results showed that the segmentation results via reflectance estimation model were improved when compared to the segmentation with sRGB images. These results suggest that it is useful to include the knowledge from illumination and observer spectral characteristics in order to increase the clustering based color image segmentation accuracy.