We introduce a new model that consists of a color classifier followed by four non-square matrices for forward characterization of display devices (i.e. predicting the XYZ value of a displayed color when the input RGB value is known). As this forward model is not directly invertible, we present a framework for developing an optimal inverse of the forward model for characterizing display devices in the opposite direction (i.e. predicting the RGB value that should be sent to a display to obtain a known XYZ value). Finally, we discuss a method to assess the discontinuity issue of these two models on the boundaries between the color classes, and a technique which we call “overlapping training” to handle this issue. Our experimental results show that these new models outperform the conventional models with a single 3×3 or 3×10 matrix.