We investigate the role of lightness perception in determining the perceived contrast of images. In particular, it is known that the background luminance of a display affects the relationship between onscreen luminance and perceived lightness. Stevens & Stevens (1963) modeled this effect using a simple power law. However, Whittle (1992) observed a more local effect, whereby subjects are more sensitive to lightness variations around the background luminance (the crispening effect). We probe lightness perception by asking subjects to manipulate the contrast of small patches on a uniform background until they appear to vary, from black to white, in a perceptually linear manner. In a second experiment, we estimate the contrast required to match the contrast of a light patch to that of a dark patch. Both experiments are conducted using five background luminance conditions, from 0 to 100% luminance. We find that subjects contrast judgments can only be modeled by first estimating the perceived lightness in a scene, using the empirically estimated lightness functions, before computing contrast. We conclude that models of contrast perception must include sophisticated models of lightness perception.