In developing copiers and printers, it is necessary to design the image gloss according to customers' preferences. In electro-photographic printers, the image gloss depends on the system parameters, such as parameter sets for fusing (pressure, temperature, and dwell time), viscoelastic property of toners, and surface properties of paper. Therefore, it is necessary to develop system models that indicate relations between the image gloss and system parameters. In this paper, we describe the development of an image gloss prediction model based on image structures that are related to the physical phenomena in the system. We have extracted two characteristic image structures from the analysis of image samples printed by various system parameters. A “micro hollow at the toner boundary” is a boundary dimple among a few toner particles, and a “macro hollow” is a pit composed of a group of toner particles. The “micro hollow at the toner boundary” is formed by a melting phenomenon of the toner particle in the fusing process, and the “macro hollow” is affected by the arrangement of the toner particles in the pre-fusing process. Furthermore, we succeeded in constructing a model that predicts the image gloss based on the sum of the area ratios (occupancy ratio in the surface area) of the two image structures.