We consider the problem of estimating surface-spectral reflectance with a smoothness constraint from image data. The total variation of a spectral reflectance over the visible wavelength range is defined as the measure of smoothness. A penalty on roughness, equivalent to smoothness, is added to the performance index to estimate the spectral reflectance functions. The optimal estimates of the spectral reflectance functions are determined to minimize a total cost function consisting of the estimation error and the roughness of the spectral functions. An RGB camera and multiple LED light sources are used to construct the multispectral image acquisition system. We model the observed images using spectral sensitivities, illuminant spectrum, unknown spectral reflectance, a gain parameter, and an additive noise term. The estimation algorithms are developed for the two estimation methods of PCA and LMMSE. The optimal estimators are derived based on the least-square criterion for PCA and the mean squared error minimization criterion for LMMSE. The feasibility of the proposed method is shown in an experiment using three mobile phone cameras. It is confirmed that the optimal estimators improve the accuracy for both original PCA and LMMS estimators.