This paper presents a method for synthesizing 2D and 3D sensor data for various machine vision tasks. Depending on the task, different processing steps can be applied to a 3D model of an object. For object detection, segmentation and pose estimation, random object arrangements are generated automatically. In addition, objects can be virtually deformed in order to create realistic images of non-rigid objects. For automatic visual inspection, synthetic defects are introduced into the objects. Thus sensor-realistic datasets with typical object defects for quality control applications can be created, even in the absence of defective parts. The simulation of realistic images uses physically based rendering techniques. Material properties and different lighting situations are taken into account in the 3D models. The resulting tuples of 2D images and their ground truth annotations can be used to train a machine learning model, which is subsequently applied to real data. In order to minimize the reality gap, a random parameter set is selected for each image, resulting in images with high variety. Considering the use cases damage detection and object detection, it has been shown that a machine learning model trained only on synthetic data can also achieve very good results on real data.
The immersive Virtual Reality (VR) environment transforms the way people learn, work, and communicate. While the traditional modeling tools such as Blender and AutoCAD are commonly used for industrial design today, handling 3D objects on a two-dimensional screen with a keyboard and mouse is very challenging. In this work, we introduce a VR modeling system named The Virtual Workshop supporting the design and manipulation of various 3D objects in virtual environments. The proposed system was developed for the Oculus Rift platform allowing a user to efficiently and precisely design 3D objects using two hands directly. The friendly system GUI supports the user to create new 3D objects from scratch using premade “basic objects” or alternatively to import an existing 3D object made in other applications. Meanwhile, the finished 3D models are stored in the standard OBJ file format and exported later for the development of 3D scenarios in other applications such as the Unity engine. From concept to design, the VR modeling system provides an open platform where the designers and the clients can better share their ideas as well as interactively refine the rendered virtual models in collaboration.
This paper presents a novel method for 3D scene modeling using stereo vision, with an application to image registration. The method constists of two steps. First, disparity estimates are refined, by filling gaps of invalid disparity and removing halos of incorrectly assigned disparity. A coarse segmentation is obtained by identifying depth slices, after which objects are clustered based on color and texture information using Gabor filters. The second step consists of reconstructing the resulting objects in 3D for scene alignment by fitting a planar region. A 2D triangle mesh is generated, and a 3D mesh model is obtained by projecting each triangle onto the fitted plane. Both of these extensions result in improved alignment quality with respect to the state of the art, and operate in near real time using multi-threading. As a bonus, the refined disparity map can also be used in combination with the existing method.