When evaluating the image quality, people mostly would like to concentrate on the color appearance of memory objects, representing the naturalness and reality of the image scene. Generally, an image with objects which have perfect memory colors reproduction will give natural and harmonious feelings. Many previous studies had proved the critical role of naturalness in image quality assessment, but it was still tough to scale the image naturalness precisely. In this study, natural images with blue sky, green grass, and skin colors were selected and partially rendered to develop the model of preference and naturalness of typical memory colors. A psychophysical experiment was conducted to collect the visual data of these images. Afterward, the psychophysical data were used to build the preference models and naturalness models, respectively. The models were then compared with previous studies. Results showed that the new models could accurately predict the preference and naturalness of target memory colors.