In this work, we present a color quantification method based on the matrix of local pallets and colorimetric criteria. The proposed method extracts a set of onedimensional colors resulting from image partitioning. Image windowing depends upon the image variance, which gives information about color dispersion. The color sets are then used to generate the rows of the local pallet matrix that will be used as a smaller image but more interesting. The selection of the principal pallet used to quantify the color image is accomplished on the local pallet matrix by computing the histogram. From this histogram we extract recursively the most important color. Then, we eliminate its n most similar colors. To avoid conflict between equi-frequent colors we use EMD distance that determines the best color by matching the results. Finally, image is quantified by replacing each pixel's color by the nearest color from the final pallet.