Color enables humans to readily extract features of an object, leading us to describe tomatoes and apples as “red” despite the presence of other colors. How observers accomplish this is not well understood. In this study, we present observers with rapidly presented stimuli at varying levels of context. Observers were asked to select the color that best represents the image from eight options. We found that observers tended to select progressively lighter or darker colors as more context was introduced, although whether the representative color choice became darker or lighter varied from image to image. This is likely a result of observers discounting achromatic cues (i.e.: specular highlights, shadows) as context is revealed, but why images were treated inconsistently requires further investigation. Observer responses were noisily distributed. These results shed light on how observers characterize the color of multicolored objects.