Abstract This article proposes a confusion-line separation algorithm in a CIELAB color space using color segmentation for protanopia and deuteranopia. Images are segmented into regions by grouping adjacent pixels with similar color information using the hue components of the images. To this end, the region-growing method and the seed points used in this method are the pixels that correspond to peak points in hue histograms. In order to establish a color vision deficiency (CVD) confusion-line map, the authors establish 512 virtual boxes in an RGB 3-D space so that boxes existing on the same confusion line can be easily identified. The authors then check whether segmented regions exist on the same confusion line and perform a color adjustment in a CIELAB color space so that all adjacent regions exist on different confusion lines in order to provide the best color-identification effect for those with CVDs.