Coral reef ecosystems are some of the diverse and valuable ecosystems on earth. They support more species per unit area than any other marine environment and are essential to the sustenance of life in our oceans. However, due to climate change, only under 46% of the worlds coral were considered healthy as of 2008. One of the biggest challenges with regard to coral conservation is that reef mapping is currently carried out manually, with a group a divers manually moving and placing a large PVC quadrat for every unit area of the reef and then photographing and analyzing each unit separately. Hence, there is a pressing need to improve the methodology of imaging, stitching and analyzing coral reef maps in order to make it feasible to protect them and sustain life in our oceans. To improve the current methodology, a reef-mapping surface drone robot which photographs, stitches and analyzes the reef autonomously was built. This robot updates the physical quadrat which is used today, to a projected laser quadrat, which eliminates the need to dive to the bottom of the sea and allows relative pose estimation. The robot then captures and processes the images and using 3D reconstruction and computer vision algorithms is able to map and classify the coral autonomously.

Inventory management and handling in warehouse environments have transformed large retail fulfillment centers. Often hundreds of autonomous agents scurry about fetching and delivering products to fulfill customer orders. Repetitive movements such as these are ideal for a robotic platform to perform. One of the major hurdles for an autonomous system in a warehouse is accurate robot localization in a dynamic industrial environment. Previous LiDAR-based localization schemes such as adaptive Monte Carlo localization (AMCL) are effective in indoor environments and can be initialized in new environments with relative ease. However, AMCL can be influenced negatively by accumulated odometry drift, and is also reliant primarily on a single modality for scene understanding which limits the localization performance. We propose a robust localization system which combines multiple sensor sources and deep neural networks for accurate real-time localization in warehouses. Our system employs a novel deep neural network architecture consisting of multiple heterogeneous deep neural networks. The overall architecture employs a single multi-stream framework to aggregate the sensor information into a final robot location probability distribution. Ideally, the integration of multiple sensors will produce a robust system even when one sensor fails to produce reliable scene information.

Towards the actualization of a disaster response robot that can locate and manipulate a drill at an arbitrary position with an arbitrary posture in disaster sites, this paper proposes a method that can estimate the position and orientation of the drill that is to be grasped and manipulated by the robot arm, by utilizing the depth camera information acquired by the depth camera. In this paper’s algorithm, first, using a conventional method, the target drill is detected on the basis of an RGB image captured by the depth camera, and 3D point cloud data representing the target is generated by combining the detection results and the depth image. Second, using our proposed method, the generated point cloud data is processed to estimate the information on the proper position and orientation for grasping the drill. More specifically, a pass through filter is applied to the generated 3D point cloud data obtained by the first step. Then, the point cloud is divided, and features are classified so that the chuck and handle are identified. By computing the centroid of the point cloud for the chuck, the position for grasping is obtained. By applying Principal Component Analysis, the orientation for grasping is obtained. Experiments were conducted on a simulator. The results show that our method could accurately estimate the proper configuration for the autonomous grasping a normal-type drill.

In this paper, we develop an automated optical inspection method to detect yarn packages’ defect. Although textile industry is regarded as traditional industry, many new technologies, e.g., computer vision detection algorithms, are applied to this industry I recent years. Yarn packages are the semi-finished good of textile industry. Various factors may cause abnormal-shaped packages. In this study, we develop three defect detection algorithms to extract abnormal-shape packages. These algorithms can help manufacturer to avoid the disadvantages of human inspection effectively and improve the productive quality.

Video-based detection of moving and foreground objects is a key computer vision task. Temporal differencing of video frames is often used to detect objects in motion, but fails to detect slowmoving (relative to the video frame rate) or stationary objects. Adaptive background estimation is an alternative to temporal frame differencing that relies on building and maintaining statistical models describing background pixel behavior; however, it requires careful tuning of a learning rate parameter that controls the rate at which the model is updated. We propose an algorithm for statistical background modeling that selectively updates the model based on the previously detected foreground. We demonstrate empirically that the proposed approach is less sensitive to the choice of learning rate, thus enabling support for an extended range of object motion speeds, and at the same time being able to quickly adapt to fast changes in the appearance of the scene.

By combining terrestrial panorama images and aerial imagery, or using LiDAR, large 3D point clouds can be generated for 3D city modeling. We describe an algorithm for change detection in point clouds, including three new contributions: change detection for LOD2 models compared to 3D point clouds, the application of detected changes for creating extended and textured LOD2 models, and change detection between point clouds of different years. Overall, LOD2 model-to-point-cloud changes are reliably found in practice, and the algorithm achieves a precision of 0.955 and recall of 0.983 on a synthetic dataset. Despite not having a watertight model, texturing results are visually promising, improving over directly textured LOD2 models.

In recent years, buildings in urban areas are frequently being demolished for a variety of reasons. For example, demolitions happen when buildings are rebuilt because the building was sold during an asset sale by a corporation in financial difficulties, because the building was built during the period of high economic growth and was aging, or because the building was damaged in a natural disaster, which happens frequently in recent years. However, construction waste is still being sorted by hand. Therefore, it is desired to reduce labor costs and to improve safety of workers. In order to overcome these issues, this study investigated methods for automatically recognizing waste materials. However, these methods had several problems, such as low recognition accuracy and an inability to handle metals, such as iron. In this research, we propose a method for automatically recognizing waste materials using sensor fusion. In the proposed method, information regarding the color, brightness, and shape of the object is acquired from images obtained using imaging sensors. In addition, we also focus on differences in the thermal conductivity of different materials and use a thermal sensor to measure the temperature of the target object to obtain thermal information. We performed a material recognition experiment in which only camera images were used, and a material recognition experiment in which sensor fusion was used. The results show that the recognition accuracy was approximately 10% higher overall in the experiment conducted using the latter method compared to the experiment conducted using the former method. These results show that the proposed method is effective.

Recently, the semantic inference from images is widely used for various applications, such as augmented reality, autonomous robots, and indoor navigation. As a pioneering work for semantic segmentation, the fully convolutional networks (FCN) was introduced and outperformed traditional methods. However, since FCN only takes account of the local contextual dependency, it does not reflect the global contextual dependency. In this paper, we explore variants of FCN with local and global contextual dependencies in the semantic segmentation problem. In addition, we tried to improve the performance of semantic segmentation with extra depth information from a commercial RGBD camera. Our experiment result indicates that exploiting the global contextual dependencies and the additional depth information improves the quality of semantic segmentation

Change detection from ground vehicles has various applications, such as the detection of roadside Improvised Explosive Devices (IEDs). Although IEDs are hidden, they are often accompanied by visible markers, which can be any kind of object. Because of this, any suspicious change in the environment compared to an earlier moment in time, should be detected. Little work has been published to solve this ill-posed problem using deep learning. This paper shows the feasibility of applying convolutional neural networks (CNNs) to HD video, to accurately predict the presence and location of such markers in real time. The network is trained for the detection of pixel-level changes in HD video, compared to an earlier reference recording. We investigate Siamese CNNs in combination with an encoder-decoder architecture and introduce a modified double-margin contrastive loss function, to achieve pixel-level change detection results. Our dataset consists of seven pairs of challenging real-world recordings with geo-tagged test objects. The proposed network architecture is capable of comparing two images of 1920×1440 pixels in 150 ms on a GTX1080Ti GPU. The proposed network significantly outperforms state-of-the-art networks and algorithms on our dataset in terms of F-1 score, on average by 0.28.