In this work, the most relevant 3D LiDAR technologies and their applications in 2022 were investigated. For this purpose, applications of LiDAR systems were classified into the typical application areas "3D modeling", "smart city", "robotics", "smart automotive" and "consumer goods". The investigation has shown that neither "mechanical" LiDAR technologies, nor so-called solid-state LiDAR technologies, nor "hybrid" LiDAR technologies can be evaluated as optimal for the typical application areas. In none of the application areas could all of the elaborated requirements be met. However, the "hybrid" LiDAR technologies such as sequential MEMS LiDAR technology and sequential flash LiDAR technology proved to be among the most suitable for most typical application areas. However, other technologies also tended to be suitable for individual typical application areas. Finally, it was found that several of the LiDAR technologies investigated are currently equally suitable for some typical application areas. To evaluate the suitability, concrete LiDAR systems - of different technologies and properties - were compared with the specific requirements of exemplary applications of an application area. The results of the investigation provide an orientation as to which LiDAR technology is promising for which application area.
Autonomous driving plays a crucial role to prevent accidents and modern vehicles are equipped with multimodal sensor systems and AI-driven perception and sensor fusion. These features are however not stable during a vehicle’s lifetime due to various means of degradation. This introduces an inherent, yet unaddressed risk: once vehicles are in the field, their individual exposure to environmental effects lead to unpredictable behavior. The goal of this paper is to raise awareness of automotive sensor degradation. Various effects exist, which in combination may have a severe impact on the AI-based processing and ultimately on the customer domain. Failure mode and effects analysis (FMEA) type approaches are used to structure a complete coverage of relevant automotive degradation effects. Sensors include cameras, RADARs, LiDARs and other modalities, both outside and in-cabin. Sensor robustness alone is a well-known topic which is addressed by DV/PV. However, this is not sufficient and various degradations will be looked at which go significantly beyond currently tested environmental stress scenarios. In addition, the combination of sensor degradation and its impact on AI processing is identified as a validation gap. An outlook to future analysis and ways to detect relevant sensor degradations is also presented.
The performance of autonomous agents in both commercial and consumer applications increases along with their situational awareness. Tasks such as obstacle avoidance, agent to agent interaction, and path planning are directly dependent upon their ability to convert sensor readings into scene understanding. Central to this is the ability to detect and recognize objects. Many object detection methodologies operate on a single modality such as vision or LiDAR. Camera-based object detection models benefit from an abundance of feature-rich information for classifying different types of objects. LiDAR-based object detection models use sparse point clouds, where each point contains accurate 3D position of object surfaces. Camera-based methods lack accurate object to lens distance measurements, while LiDAR-based methods lack dense feature-rich details. By utilizing information from both camera and LiDAR sensors, advanced object detection and identification is possible. In this work, we introduce a deep learning framework for fusing these modalities and produce a robust real-time 3D bounding box object detection network. We demonstrate qualitative and quantitative analysis of the proposed fusion model on the popular KITTI dataset.
Modern warehouses utilize fleets of robots for inventory management. To ensure efficient and safe operation, real-time localization of each agent is essential. Most robots follow metal tracks buried in the floor and use a grid of precisely mounted RFID tags for localization. As robotic agents in warehouses and manufacturing plants become ubiquitous, it would be advantageous to eliminate the need for these metal wires and RFID tags. Not only do they suffer from significant installation costs, the removal of wires would allow agents to travel to any area inside the building. Sensors including cameras and LiDAR have provided meaningful localization information for many different positioning system implementations. Fusing localization features from multiple sensor sources is a challenging task especially when the target localization task’s dataset is small. We propose a deep-learning based localization system which fuses features from an omnidirectional camera image and a 3D LiDAR point cloud to create a robust robot positioning model. Although the usage of vision and LiDAR eliminate the need for the precisely installed RFID tags, they do require the collection and annotation of ground truth training data. Deep neural networks thrive on lots of supervised data, and the collection of this data can be time consuming. Using a dataset collected in a warehouse environment, we evaluate the performance of two individual sensor models for localization accuracy. To minimize the need for extensive ground truth data collection, we introduce a self-supervised pretraining regimen to populate the image feature extraction network with meaningful weights before training on the target localization task with limited data. In this research, we demonstrate how our self-supervision improves accuracy and convergence of localization models without the need for additional sample annotation.
Inventory management and handling in warehouse environments have transformed large retail fulfillment centers. Often hundreds of autonomous agents scurry about fetching and delivering products to fulfill customer orders. Repetitive movements such as these are ideal for a robotic platform to perform. One of the major hurdles for an autonomous system in a warehouse is accurate robot localization in a dynamic industrial environment. Previous LiDAR-based localization schemes such as adaptive Monte Carlo localization (AMCL) are effective in indoor environments and can be initialized in new environments with relative ease. However, AMCL can be influenced negatively by accumulated odometry drift, and is also reliant primarily on a single modality for scene understanding which limits the localization performance. We propose a robust localization system which combines multiple sensor sources and deep neural networks for accurate real-time localization in warehouses. Our system employs a novel deep neural network architecture consisting of multiple heterogeneous deep neural networks. The overall architecture employs a single multi-stream framework to aggregate the sensor information into a final robot location probability distribution. Ideally, the integration of multiple sensors will produce a robust system even when one sensor fails to produce reliable scene information.