An isolated light has a color appearance specified reasonably well by its wavelength, but the same light within a complex image can appear a quite different hue. How does the context of an image affect the appearance of an embedded light? A classical approach is to aggregate light from throughout the image to determine an equivalent uniform background that has the same effect as the complex stimulus. Several models have been proposed to determine this ‘equivalent background’, including simple averaging of light, spatial weighting, and nonlinear neural responses. The main point of this paper is that none of these models can succeed because variation within a complex image is a fundamental property of it. Studies show that color perception depends on a chromatic-contrast gain-control mechanism, which is regulated by chromatic variation over a large area. Any uniform field has no variation, of course, so cannot mimic the color shifts caused by a complex image.