This paper compares two popular psychophysical scaling methods, the method of rank order and the method of paired comparison, for measuring reproduced image color quality. Although there are reports supporting that the two methods produced virtually the same results when applied to some psychophysical scaling tests, given the complexity of image color quality perception, it is of interest to investigate the validity of the relationship between the two measurement methods. In this paper, an experiment was designed using three groups of color images to examine the potential relationship between the two methods. The results prove that the two methods produce similar but different results. Methods for deriving scaled values based on rank order data are also investigated. It is concluded that simpler data processing methods can give acceptable results. Some of the advantages, disadvantages, and error sources are also discussed.