Shadow detection is undergoing active research because it plays an important role in scene understanding, and has a wide range of applications including household robots and autonomous cars. In this effort, we present a novel approach to detect cast shadows on 3D point clouds. Point Cloud Library (PCL) is used to perform plane detection on point clouds. A Markov Random Field (MRF) is then constructed on the detected plane region, with an energy term that combines plane labels, depth cues and brightness cues. The resulting system is tested against USC Shadow, a dataset we collected in a controlled environment, as well as selected scenes from NYU Depth, a dataset that contains 1449 RGB-D images of various indoor scenes. Our system shows very stable performance even on complicated scenes and heavily textured planes.