The aim of the study was to create an improved colorimetric and broadband spectral characterization for scanners and cameras. In such characterization, selecting an adequate number of color samples of known reflection spectra is necessary. And though countless sample data sets are available, the properties required of a data set for such optimal characterization remain elusive. Therefore a new methodology was required to address the characterization task. Such a characterization method is introduced in this article and is based on statistical classification of the colorimetric and broadband spectral properties of color sample sets. It introduces and effectively utilizes both the reflectance spectrum of the color sample and the spectral power distribution of the source. However it is shown that characterization methods based on a regression model can be used only if the conditions of the regression model are satisfied and that most statistical estimation errors are caused by conditions of the regression model not being satisfied (for instance heteroscedasticity, autocorrelation, multicollinearity). Nevertheless, the method introduced selects optimal representative color samples, so that with these samples the spectral responsivity of the detector can be estimated more precisely. The selection method is self-adaptive. If the reflectance spectra of the color samples and the spectral power distribution of the source are known, the optimal number of color samples, the number of principal eigenvectors, etc., are automatically set up according to the given a priori information, and the responsivity curves are determined where, the given z target function [see Eq. (5)] is minimal. The study has shown that the estimation error of broadband characterization can be decreased significantly if an optimal set of color samples is selected using these statistical methods. If there is more a priori information (for instance the spectral power distribution of the source of the scanner) the estimation error can be further decreased.