In this article, we present a U-Net convolutional network for solving insufficient data problems of color patches in colorimetric characterization. The U-Net network uses data augmentation annotated over 6,885,222 colors, 32,027,200 color patches, and 2,098 billion pixels directly from only eight standard colorimetric images of ISO 12640 (CIELAB/SCID). By applying the U-Net network trained on big augmented data, the pixel-wise colorimetric characterization is implemented from digitalized red, green, blue image samples to ISO 12640 (CIELAB/SCID) CIELAB standard colorimetric images. The performance efficiency of the U-Net network is superior to that of the convolutional neural network on both training and validating epochs. Moreover, pixel-wise color colorimetric characterization is achieved using the intelligent machine vision of U-Net integrated with a data augmentation technique to overcome the drawback of complex color patches and labor-intensive tasks. This study might improve colorimetric characterization technology with a resolution of 2560-by-2048 for over 4 million pixels. The results reveal that U-net with pixel-wise regression enhances the precise colors of images, taking detail and realism to a new level.