Predicting scene depth (or geometric information) from single monocular images is a challenging task. This paper addresses such challenging and essentially ill-posed problem by regression on samples for which the depth is known. In this regard, we first retrieve semantically similar RGB and depth pairs from datasets using a deep convolutional activation feature. We show that our framework provides a richer foundation for depth estimation than existing hand-craft representations. Subsequently, an initial estimation is then integrated by block-matching and robust patch regression. It assigns perceptually appropriate depth values to an input query in accordance with a data-driven depth prior. A final post processor aligns depth maps with RGB discontinuities, resulting in visually plausible results. Experiments on the Make 3D and NYU RGB-D datasets show competitive results compared to recent state-of-the-art methods.