This paper presents a new method for no reference mesh visual quality assessment using a convolutional neural network. To do this, we first render 2D images from multiple views of the 3D mesh. Then, each image is split into small patches which are learned to a convolutional neural network. The network consists of two convolutional layers with two max-pooling layers. Then, a multilayer perceptron (MLP) with two fully connected layers is integrated to summarize the learned representation into an output node. With this network structure, feature learning and regression are used to predict the quality score of a given distorted mesh without the availability of the reference mesh. Experiments have been successfully conducted on LIRIS/EPFL generalpurpose database. The obtained results show that the proposed method provides good correlation and competitive scores comparing to some influential and effective full and reduced reference methods.

Holostream is a novel platform which enables high-quality 3D video communication on mobile devices (e.g., iPhones, iPads) using existing standard wireless networks. The major contributions are: (1) a novel high-quality 3D video compression method that drastically reduces both 3D geometry and color texture data sizes in order to transmit them within the bandwidths provided by existing wireless networks; (2) a novel pipeline for 3D video recording, encoding, compression, decompression, visualization and interaction; and (3) a demonstration system which successfully delivered video-rate, photorealistic 3D video content through a standard wireless network to mobile devices. The novel platform improves the quality and expands upon the capabilities of popular applications already utilizing real-time 3D data delivery, such as teleconferencing and telepresence. This technology could also enable emerging applications which may require highresolution, high-accuracy 3D video data delivery, such as remote robotic surgery and telemedicine.

We present a method for synchronizing three-dimensional (3D) point cloud from 3D scene with estimation using a 3D Lidar and an RGB camera. These 3D points sensed by the 3D Lidar are not captured at the same time, which makes it difficult to measure the correct shape of the object in a dynamic scene. In our method, we generate synchronized 3D points at arbitrary times using linear interpolation in four-dimensional space, time and space. For interpolating the 3D point, we obtain corresponding 3D point matching with the pixel value captured by the RGB camera in a continuous frame. The experimental results demonstrate the effectiveness of the presented method by depicting a synchronized 3D point cloud that is correctly shaped.

Towards the actualization of an automatic guitar teaching system that can supervise guitar players, this paper proposes an algorithm for accurately and robustly tracking the 3D position of the fretboard from the video of guitar plays. First, we detect the SIFT features within the guitar fretboard and then match the detected points using KD-tree searching based matching algorithm frame by frame to track the whole fretboard. However, during the guitar plays, due to movements of the guitar neck or occlusions caused by guitar players' fingers, the feature points on the fretboard cannot always be matched accurately even though applying traditional RANSAC homography. Therefore, by using our modified RANSAC algorithm to filter out the matching error of the feature points, perspective transformation matrix is obtained between the correctly matched feature points detected at the first and other frames. Consequently, the guitar neck is tracked correctly based on the perspective transformation matrix. Experiments show promising results such as high accuracy: the total mean tracking error of only 4.17 mm and variance of 1.5 for the four tracked corners of the fretboard. This indicates the proposed method outperforms related tracking works including state-of-art Fully-convolutional Network

Hand gesture recognition is a crucial but challenging task in the field of Virtual Reality (VR) and Human Computer Interaction (HCI). In this paper, a skeleton-based dynamic hand gesture recognition approach is proposed, in which the skeleton structure of the hand captured by 3D depth sensor is firstly exploited and the spatiotemporal multi-fused features that concatenate four skeleton hand shape features and one hand direction feature are extracted. Then the hand shape features are encoded by Fisher Vector obtained from a Gaussian Mixture Model (GMM). To add the temporal information, hand shape Fisher Vector and hand direction feature are represented by a Temporal Pyramid (TP) to obtain the final feature vectors to be fed into a linear SVM classifier to recognize. The proposed approach is evaluated on a challenging dataset containing eight gestures performed by ten participants. Compared with the state-of-the-art dynamic hand gesture recognition methods, the proposed method shows a relative high recognition accuracy of 90.0%.

In the process of digitization of cultural heritage objects with differentiated shininess it is difficult to reproduce faithfully the aesthetic of the original. The aim of the presented research is to address simultaneous capturing of shape, color and reflection features in order to digitally reproduce the appearance of the real object. We focus our work on a study of a ceramic furnace tile which exhibits complex shape, color and varying reflection properties. To achieve the goal we use a specially designed automated acquisition setup and provide a dedicated data processing pipeline. The collected geometry conforms to metrological uncertainty validation and the diffuse component is colorimetrically calibrated. The reflection properties are measurement-based, modeled with Blinn-Phong and visualized with an OpenGL shader. Close integration of capturing devices and a single data processing pipeline allows to fully utilize multidimensional raw data in order to get faithful final appearance model.