This paper considers appearance reconstruction of 3D fluorescent objects with mutual illumination effect. We examined the mutual illumination between flat plane and curved objects, by placing a cylinder on a flat plane. First, the spectral images of the fluorescent objects with the mutual illumination effect are acquired under different illumination directions. The respective observed images are then decomposed into several components on the basis of spectral compositions and the spatial distributions of geometric factors. The surface normal vectors are estimated by utilizing the photometric stereo method. Second, the reference geometric factors are defined as geometric factors independent of illumination direction change. The reference geometric factors for mutual illumination on the plane are determined based on viewing from the normal direction of the cylinder. We reconstruct a new appearance of the two fluorescent objects under the fixed viewpoint but the different conditions of fluorescent material, illuminant spectrum, and illumination direction. We show that the total appearance of the reconstructed image predicts well the real scene.