Structured light depth sensors work by projecting a codeword pattern, usually made up of NIR light, on a scene and measuring distortions in the light received on an NIR camera to get estimates of the camera-projector disparities. A well-known challenge associated with using structured light technology for depth estimation is its sensitivity to NIR components in the ambient illumination spectrum. While various methodologies are employed to increase the codeword-to-ambient-light ratio – for instance, using narrow-band NIR filters and selecting a spectral band for the NIR laser where the interference from ambient light is expected to be low – structured light setups usually do not work well outdoors under direct sunlight. The standard deviation of shot noise increases as the square root of the ambient-light intensity, reducing the SNR of the received codeword pattern and making the decoding process challenging. One way to improve the SNR of the received structured light pattern is to use codewords of larger spatial support for depth sensing. While large codewords do improve the SNR of the received pattern, the disadvantage is decreased spatial resolution of the estimated disparity field. In this paper, we use a multiscale random field (MSRF) to model the codeword labels and use a Bayesian framework, known as sequential MAP (SMAP) estimation, developed originally for image segmentation, for developing a novel multiscale matched filter for structured light decoding. The proposed algorithm decodes codewords at different scales and merges coarse-to-fine disparity estimates using the SMAP framework. We present experimental results demonstrating that our multiscale filter provides noise-robust decoding of the codeword patterns, while preserving spatial resolution of the decoded disparity maps.
Light-field cameras capture 4-dimensional spatio-angular information of the light field. They provide more helpful multiple viewpoints or sub-apertures for visual analysis and visual understanding than traditional cameras. Optical flow is a common method to get scene structure cues from two images, however, subpixel displacements and occlusions are two inevitable challenges in the optical flow estimation from light-field sub-apertures. In this paper, we develop a light-field flow model, and propose an edge-aware light-field flow estimation framework for joint depth estimation and occlusion detection. It consists of three steps: i) An optical flow volume with sub-pixel accuracy is extracted from sub-apertures by edge-preserving interpolation. Then occlusion regions are detected through consistency checking. ii) Robust light-field flow and depth estimation are initialized by a winner-take-all strategy and a weighted voting mechanism. iii) Final depth map is refined by a weighted median filter based on guided filter. Experimental results demonstrate the effectiveness and robustness of our method.
In this paper, we present a new real-time depth estimation method using the stereo color camera and the ToF depth sensor. First, we obtain the initial depth information from the ToF depth sensor. Exploiting the initial depth information to narrow the disparity range by performing 3-D warping from the position of the ToF camera to the position of the stereo camera due to accelerating the algorithm. We construct the cost volume by calculating intensity difference and truncated absolute difference of gradients. After narrowing the disparity range, we aggregate the cost volume. Experimental results show that the proposed method can represent the disparity detail and improve the quality in the vulnerable areas of stereo matching.
Generating a disparity map has been a challenging issue for several decades. To improve the quality of estimated disparity map and reduce the computational complexity, efficient cost matching functions and cost aggregation methods have been developed. Especially, in case of sequential stereo matching procedure, computational complexity causes a problem in terms of the real time processing. To overcome this problem, we propose a temporal domain stereo matching method using the guided image filtering. The advantage of temporal stereo matching method is restricting a disparity search range while calculating a matching cost value along the horizontal pixel line. Additionally, we adopt the guided image filtering to improve the quality of estimated disparity map in updating procedure. Since the guided image filtering aggregates the cost value by considering object boundary region, the result of stereo matching accuracy is improved than conventional temporal stereo matching method. From the experiment results, we check that the proposed method generates the most accurate disparity map than conventional method.