This paper focuses on depth map boundary reconstruction by providing a novel goal-oriented Time-of-Flight depth map superresolution approach. State-of-the-art RGB-guided depth map upscaling uses accompanied high-resolution RGB information to upscale depth maps while leaving a minor but non-negligible amount of blurred flying pixels which are physically incorrect in depth maps. These flying pixels highly deteriorate consecutive applications which need a high-definition depth key. Our approach evaluates RGB superpixel segmentation principles to precisely indicate RGB boundaries and to finally transfer this boundary information to the depth map by assigning the maximumlikelihood estimate for each region. Thereby, we achieve discontinued and physically correct depth edges with noticeably reduced flying pixel artifacts. The proposed method overcomes previous algorithms by more than 50% in our dedicated evaluation on real RGBD camera data. This highly accurate depth edge information can be used in future applications relying on depth-based keying.