Hundreds of large color differences, of magnitude 20 ΔE₀₀, were generated and used in a visual sorting experiment. The process of generating these color differences and two specific experiments are described in detail. The results show that small color difference metrics, such as ΔE₀₀, do not consistently model the visually sorted differences for large differences. A new similarity measure, based on a cosine similarity between categorical vectors of colors, is described and used to more consistently model large color differences. This similarity metric can be used to better characterize large color errors during reproduction, for image processing operations such as segmentation or as a feature for content retrieval. The new measure can also be applied to visual phenomena, such as categorical perception, in which within category color differences are perceived as smaller than across category differences.