Color management is an indispensable technology for working with color, as it ensures that colors are maintained as much as possible when images such as photographs are represented on different devices, such as cameras, displays, and printed matter. Profile connection space (PCS) is defined by the CIELAB and CIEXYZ color systems, which are defined by the International Commission on Illumination (CIE). The L*a*b* and XYZ color spaces are called device-independent colors because they reproduce the same color no matter what device is used to output the image as long as the values are identical. On the other hand, RGB and CMYK, which are generally used for color reproduction, are called device-dependent colors because different colors are reproduced on different output devices even if the numerical values are the same [1].

Figure 1 shows the workflow for creating an image for printing [2]. In the printing industry, the first step in printing is photographing and development by a photographer, followed by color conversion and color-tone correction, plate making, and printing, in that order. To construct an optimal workflow, it is necessary to standardize the upstream photography process and the downstream printing process, respectively. The standardization of imaging and printing can be said to be the standardization of colors in RGB and CMYK. However, as shown in Figure 2, the color-reproduction areas of RGB and CMYK are different, making it difficult to standardize color conversion. Furthermore, in such a process, customers often judge the quality of the color based on sensory criteria, and to meet their sensory demands, the color tone must be corrected many times in the plate-making process. Since colors are managed subjectively rather than by objective numerical values, the reproducibility and stability of colors are not maintained.

To promote printing standardization, some printing companies have introduced the Japan Color reference color and certification standards, which can quantitatively express colors, as color management standards [3]. This enables accurate evaluation of printed materials and numerical management that is independent of differences in the skills and experience of printing operators. In some cases, when color conversion is performed, the color is not accurately reproduced, but is built in so that it is the optimal color when printed. For example, skin tones are corrected to be brighter and less muddy than the actual colorimetric values by increasing the brightness and saturation, and the color of cherry blossoms is corrected to be closer to pink rather than white as the human eye perceives it. This is because the color that a person remembers in an image is called a memory color (impression color), which may differ from the actual color captured by a camera or other means. Preferred color reproduction involves human memory, and since humans tend to store colors in their brains as more vivid than they are, there are differences between the remembered and recorded colors [4]. However, the mechanism of desirable color reproduction involves higher-order information processing in the brain, which has not yet been elucidated. In addition to the enormous amount of time required for this kind of correction work, which requires capturing the characteristics of each image one by one, it is also a task filled with empirical know-how, making it difficult to pass on the skills to the next generation.

In this study, therefore, we propose color-correction skills in the printing industry through deep learning. We built an automatic color-correction model by learning from a large database of color-converted images that has been developed by printing companies.

The remainder of this paper is structured as follows, Section 2 explains related works, which are similar to generative adversarial network (GAN). Section 3 explains our approach, which consists of dataset construction, and how to evaluate learning models. The results of the quantitative and qualitative evaluation are discussed in Section 4. Section 5 presents the conclusions and future works.

In this section, we introduce some research about GANs [5], one of which is used in the proposed approach. Research on GANs has been active, and there are many derivatives of the original GAN. Among them, those related to the original research on which this method is based are mentioned below.

Conditional GAN (CGAN), a network proposed by Mirza et al. in 2014 [6] that follows the basic structure of GANs using generators and discriminators but is extended to perform learning conditionally by providing additional information. The basic structure of GANs using generators and discriminators is retained. While ordinary GANs could only use noise vectors as input to the generator, CGAN has been improved to allow the input of condition data corresponding to the condition vectors to the discriminator.

Pix2pix is an image transformation method based on CGAN, proposed by Isola et al. as an image-generation algorithm [7]. Therefore, it is not possible to control the generated data by specific conditions such as categories. Pix2pix is a type of CGAN in which the generator takes the input image x as input and generates an image that is close to the correct image y. The discriminator, on the other hand, identifies whether the input is a pair of the generator image G(x) and the input image x or a pair of the correct image y and the input image x.

The generator employs a U-Net-based network [8], which consists of encoder path and decoder paths. In addition, skip connections are introduced between each of them to transfer feature maps at each layer in the encoder path to the same depth layer in the decoder path. The feature map at each layer in the encoder path is transmitted to a layer of the same depth in the decoder path. This enables learning without losing detailed information in the image. The encoder passes extract local features of the input image while down-sampling by convolution, and the decoder pass reconstructs the image based on the feature map while up-sampling by inverse convolution.

PatchGAN architecture is used as the discriminator [9]. PatchGAN divides the entire image into patches of fixed size and determines the truth or falsity of each patch. Finally, the output of all blocks is averaged, and the average value is used as the final output. This provides validity for detailed parts of the image. The network is a fully convolutional network, which consists of only convolutional layers, where the input is the entire image and the output is a feature map consisting of the results of the patch judgments. Pix2pix is an effective network for image transformation and has been studied for various images. It can be directly applied to real image transformation from labeling information, colorization of black-and-white images, land use analysis, and detection of geospatial elements.

There are some approaches for image-to-image translation, the most famous being pix2pix, cycleGAN [10] and adversarial inverse graphics networks [11]. The image sizes in adversarial inverse graphics networks are 128 × 128, while the sizes of the other two are 256 × 256. In addition, pix2pix is better at directly converting images [12]. These are the reasons we used pix2pix in this approach.

Not only pix2pix, but a lot of studies have also contributed to image transformation. Luan et al. proposed a method for style transfer of images and allowed input images to be transferred to the same style as the target images [13]. Also, Huang and Belongie used adaptive instance normalization to realize style transfer in real time, which enables more flexible style transfer by linking the feature map of input images to style images [14].

In this study, we proposed a method based on pix2pix to learn and automate techniques for correcting color changes caused by the conversion from RGB to CMYK using a color-transformation image database. The effectiveness of automatic color correction was demonstrated by the proposed method.

Future works include improving the accuracy of local features in the image and extending the image size. One problem with the current learning results is that the correction tends to be applied in the same direction and with the same strength to the entire image. Manual correction, on the other hand, considers various objects and backgrounds in the image and applies the appropriate correction for each of them. We believe that image transformation using deep learning, by introducing object recognition features and recognizing each object and color in the image, and deepening learning tailored to each of them, will make it possible to correct images according to local features of the image.

In addition, the proposed method can only process images of size 256 × 256 pixels. However, considering practical situations, image sizes vary, and we consider it a prerequisite to be applicable to larger images. Pix2pixHD [23], an extension of pix2pix, achieves image transformation for high-resolution images of 2048 × 1024 pixels. We would like to consider applying such a model to improve the size of images that can be learned.

Furthermore, the images in all datasets in this study were compressed to 256 × 256 pixels, resulting in low-resolution and blurred images. This caused some of the images to be corrupted during image processing owing to the reduced resolution. Therefore, it is necessary to construct a model that can be applied to high-resolution images, as described above, or to use networks that restore low-resolution images to high-resolution, such as SRCNN [24] and GAN for super-resolution [25], in response to the results of this study.