Most digital cameras today employ Bayer Color Filter Arrays in front of the camera sensor. In order to create a true-color image, a demosaicing step is required introducing image blur and artifacts. Special sensors like the Foveon X3 circumvent the demosaicing challenge by using pixels lying on top of each other. However, they are not commonly used due to high production cost and low flexibility. In this work, a multi-color multi-view approach is presented in order to create true-color images. Therefore, the red-filtered left view and the blue-filtered right view are registered and projected onto the green-filtered center view. Due to the camera offset and slightly different viewing angles of the scene, object occlusions might occur for the side channels, hence requiring the reconstruction of missing information. For that, a novel local linear regression method is proposed, based on disparity and color similarity. Simulation results show that the proposed method outperforms existing reconstruction techniques by on average 5 dB.
We propose a deep learning method to retrieve the most similar 3D well-designed model that our system has seen before, given a rough 3D model or scanned 3D data. We can either use this retrieved model directly or use it as a reference to redesign it for various purposes. Our neural network consists of 3 different neural networks (sub-nets). The first neural network deals with object images (2D projection) and the other two deals with voxel representations of the 3D object. At the last stage, we combine the results of all 3 sub-nets to get the object classification. Furthermore, we use the second to last layer as a feature map to do the feature matching, and return a list of top N most similar well-designed 3D models.