3D shape reconstruction is one of the most important topics in computer vision and the foundation for a wide field of application. Among various technologies, structured light is one of the most reliable techniques. However, given the field of view of projectors and cameras available in the market, the working distance needed for projectors is typically larger than that for cameras. To reduce the working distance of the projectors while covering the whole working platform, two projectors with their field of view overlapping are used to cover the working area which holds objects to be scanned. We present a spectral analysis based model for the projector-camera system, in order to find the most distinguishable colors for two projectors, and best separate the projected patterns from two projectors. The optimal values of the two colors are determined by the pattern search method in the presence of noise, which is modeled as multivariate Gaussian noise, and characterized for different input colors. The camera sensors' responses to the projector are measured after linearization with gray balance curves. After being properly calibrated, based on one image shot of the object with binary M-array patterns projected on it, the system can reconstruct a 3D shape of the object surface.
Structured light depth sensors work by projecting a codeword pattern, usually made up of NIR light, on a scene and measuring distortions in the light received on an NIR camera to get estimates of the camera-projector disparities. A well-known challenge associated with using structured light technology for depth estimation is its sensitivity to NIR components in the ambient illumination spectrum. While various methodologies are employed to increase the codeword-to-ambient-light ratio – for instance, using narrow-band NIR filters and selecting a spectral band for the NIR laser where the interference from ambient light is expected to be low – structured light setups usually do not work well outdoors under direct sunlight. The standard deviation of shot noise increases as the square root of the ambient-light intensity, reducing the SNR of the received codeword pattern and making the decoding process challenging. One way to improve the SNR of the received structured light pattern is to use codewords of larger spatial support for depth sensing. While large codewords do improve the SNR of the received pattern, the disadvantage is decreased spatial resolution of the estimated disparity field. In this paper, we use a multiscale random field (MSRF) to model the codeword labels and use a Bayesian framework, known as sequential MAP (SMAP) estimation, developed originally for image segmentation, for developing a novel multiscale matched filter for structured light decoding. The proposed algorithm decodes codewords at different scales and merges coarse-to-fine disparity estimates using the SMAP framework. We present experimental results demonstrating that our multiscale filter provides noise-robust decoding of the codeword patterns, while preserving spatial resolution of the decoded disparity maps.