In this paper, we propose an approach based on a hidden Markov model (HMM) which accounts for the structure of a spectrum for unsupervised image segmentation. We compute feature sequences representing spectra for training and testing the HMM. In our study, each HMM models one class of a spectral color image. The algorithm computes the model parameters and the discrimination information between the HMMs. The experiments show that the proposed approach gives promising results.