Spectral imaging has been widely developed over the last ten years for archiving cultural heritage. It can retrieve spectral reflectance of each scene pixel and provide the possibility to render images for any viewing condition. A new spectral reconstruction method, the matrix R method, can achieve high spectral and colorimetric accuracies simultaneously for a specific viewing condition. Although the matrix R method is very effective, the reconstructed reflectance spectrum is not smooth when compared with in situ spectrophotometry. The goal of this research was to smooth the spectrum and make it more accurate. One possible solution is to identify pigments and find their compositions for each pixel. After that, the reflectance spectrum can be modified based on two-constant Kubelka-Munk theory using the absorption and scattering coefficients of these pigments, weighted by their concentrations. The concentrations were optimized to best fit the spectral reflectance predicted by the matrix R method. As a preliminary experiment, it was assumed that a custom target was painted using several known pigments. The simulation results show that incorporating pigment mapping into the matrix R method can recover the smoothness of the reflectance spectrum, and further improve spectral accuracy of spectral imaging.