The objectives of this study were to develop a specific visual dataset comprising samples with low lightness (L* range from 11 to 19), covering near neutral black substrates that varied in hue and chroma, and testing the performance of major color difference formulae currently in use as well as more recent CIECAM02 color difference formulae including CAM02-LCD, CAM02-SCD as well as CAM02-UCD models based on black samples. The dataset comprised 50 dyed black fabrics with a distribution of small color differences from 0 to 5. The visual color difference between each sample and the standard was assessed by 20 observers in three separate sittings with an interval of at least 24 hours between trials using an AATCC standard gray scale and a total of 3000 assessments were obtained. A third-degree polynomial equation was used to convert gray scale ratings to visual differences. The Standard Residual Sum of Squares (STRESS) was employed to evaluate the performance of thirteen color difference formulae based on visual results. Based on the analysis of STRESS index results the CAM02-SCD color difference equation performed better than other equations, however, all equations performed poorly in this region of the color space.