As VR technology has advanced, its use in performance-critical fields such as medical training and vision research has grown, driving a need for increasingly realistic VR environments. In previous work, we evaluated lightness constancy in a task where viewers matched the reflectance of surfaces at different 3D orientations, and we found substantially poorer lightness constancy in VR than in a physical apparatus. Poor constancy in VR may have been due to simplified rendering of scenes in that study, e.g., largely achromatic Lambertian surfaces. Motivated by these findings, here we evaluated lightness constancy in more realistic VR scenes, rendered with a broad array of materials, colors, textures, and specular highlights, as well as more realistic shadows. We tested two conditions: a Full-Context condition, where these lighting and material cues were available, and a Reduced-Context condition, where they were not. Participants had significantly better lightness constancy in the Full-Context condition than in the Reduced-Context condition, indicating that they exploited these additional cues. However, lightness constancy was still quite poor in absolute terms, despite the availability of rich lighting and material cues. The reasons for this failure of constancy are unclear from previous literature, and this finding suggests a promising research problem with both fundamental interest and practical applications.