This research studies the influence of the color model used for the optimal sample selection from a larger dataset, for the task of color camera calibration. Most algorithms proposed in the literature perform such sample selection in the CIELAB color space, however since color transformations from one space to another are non-linear, choice of color space will affect the final target design. This work establishes the relationship between the color space where the patches are selected and the accuracy of the final calibration result. The Kennard-Stone algorithm with different distance metrics is used to choose a fixed number of patches for the calibration target. Final calibration results are compared for various selections of patches from three datasets, using different state of the art color camera calibration methods. The results are also compared to those obtained with a classical Macbeth Color Checker. This research highlights the importance of setting proper parameters for color patches selection for custom calibration target design.