A single-camera one-shot multispectral imaging system with multispectral filter array (MSFA) has the potential to promote fast and low-cost multispectral imaging. Because the restoration accuracy of the image acquired with this imaging system depends on MSFA, optimization for MSFA has been demanded. Conventional optimizing methods have designed the versatile MSFA without dependence on the imaging object. In the field that requires multispectral images, an imaging object is often determined before designing MSFA. If MSFA can be designed based on the imaging object, the restoration accuracy becomes higher than the versatile MSFA of the conventional methods. In this paper, we propose an optimizing method for MSFA that uses the training data of the imaging object. The proposed method optimizes the observation wavelength and filter arrangement based on simulated annealing. The experiment results demonstrate that the proposed method outperforms conventional methods quantitatively.