As digital imaging becomes more widespread in a variety of industries, new standards for measuring resolution and sharpness are being developed. Some differ significantly from ISO 12233:2014 Modulation Transfer Function (MTF) measurements. We focus on the ISO 16505 standard for automotive Camera Monitor Systems, which uses high contrast hyperbolic wedges instead of slantededges to measure system resolution, defined as MTF10 (the spatial frequency where MTF = 10% of its low frequency value). Wedges were chosen based on the claim that slanted-edges are sensitive to signal processing. While this is indeed the case, we have found that wedges are also highly sensitive and present a number of measurement challenges: Sub-pixel location variations cause unavoidable inconsistencies; wedge saturation makes results more stable at the expense of accuracy; MTF10 can be boosted by sharpening, noise, and other artifacts, and may never be reached. Poor quality images can exhibit high MTF10. We show that the onset of aliasing is a more stable performance indicator, and we discuss methods of getting the most accurate results from wedges as well as misunderstandings about low contrast slanted-edges, which correlate better with system performance and are more representative of objects of interest in automotive and security imaging.

In this paper, we present framework for visualization of the vehicle-surround-views that include multiple cameras attached to the car exterior. The proposed framework harmonizes the input camera images in terms of brightness, colour and other related properties to enable advanced visualization, where the displayed image looks as it would have been captured by a single camera positioned at an arbitrarily chosen 3D point and oriented in 3D space around the vehicle. The rendering and harmonization framework is a hybrid based scheme that performs both adaptive camera tuning and post processing of the camera images. We discuss both algorithmic and implementation aspects of the image quality module within which the framework is designed. The algorithms involved in the image quality module perform camera image processing, which includes both image analyses and image post-tuning. In addition to algorithmic aspect of the framework, this paper also discusses real-time implementation aspects related to different embedded systems presently used in automotive systems.

In this paper, we propose a new method for accelerating stereo matching in autonomous vehicles using an upright pinhole camera model. It is motivated by that stereo videos are more restricted when the camera is fixed on the vehicles driving on the road. Assuming that the imaging plane is perpendicular to the road and the road is generally flat, we can derive the current disparity based on the previous one and the flow. The prediction is very efficient that only requires two multiplications per pixel. In practice, this model may not hold strictly but we still can use it for disparity initialization. Results on real datasets demonstrate the our method reduces the disparity search range from 128 to 61 with only slightly accuracy decreasing.

In modern vehicles bird's view systems are widely used to show the direct car surroundings to the driver. However, state-of-the-art methods for bird's view computations suffer from heavy distortions and unnatural warping. We propose an approach towards perspectively correct bird's view images for vehicular applications. Our method uses stereo images as input and is tested using stereo datasets.

As applications of drone proliferate, it has become increasingly important to equip drones with automatic sense and avoid (SAA) algorithms to address safety and liability concerns. Sense and avoid algorithms can be based upon either active or passive sensing methods. Each of them has advantages when compared to the other but neither is sufficient by itself. Therefore, especially for application such as autonomous navigation where failure could be catastrophic, deploying both passive and active sensors simultaneously and utilizing inputs from them become critical to detect and avoid objects in a reliable way. As part of the solution, in this paper, we present an efficient SAA algorithm based on input from multiple stereo cameras, which can be implemented on a low-cost and low-power embedded processor. In this algorithm, we construct an instantaneous 3D occupancy grid (OG) map at each time instance using the disparity information from the stereo cameras. Then, we filter noise using spacial information, and further filter noise using a probabilistic approach based on temporal information. Using this OG Map, we detect threats to the drone in order to determine the best trajectory for it to reach a destination.

With cars driving autonomously on roads, functional safety assumes critical importance to avoid hazardous situations for humans in the car and on the road. ISO 26262 defines Automotive Safety Integration Level (ASIL) with level QM (Least) to ASIL-D (Highest) based on severity and probability of defect causing harm to human life. This paper explores functional safety requirements and solutions for software systems in autonomous cars in four broad aspects. The first aspect covers usage of redundancy at various levels to ensure the failure of one system does not affect the overall operation of the car. It explores the usage of redundancy via multiple sensors and diverse processing of data to arrive at functionally safe results. Based on the redundancy requirements, in the second aspect, an HW (SoC) and SW architecture is proposed which can help meet these requirements. It explores the definition of SW framework, task scheduling, and tools usage to ensure systematic faults are prevented at the development stage. Autonomous driving systems will be complex and expecting all software modules comply with the highest functional safety level may not be feasible. The third aspect explores the usage of freedom from interference (FFI) via HW and SW mechanisms like Firewalls, MMU to allow safe and non-safe sub-systems to co-exist and operate according to their specification. The final aspect covers usage of SW and HW diagnostics to monitor, detect, and correct random faults found at run-time in HW modules. It explores the usage of diagnostics features like ECC, CRC, and BIST to help detect and avoid runtime failures.

Over two million people in the United States rely on the use of a wheelchair to perform daily tasks. Joystick controls on motorized wheelchairs have improved the lifestyles of so many, but are of little value to the visually impaired or patients with restricted hand mobility. Often times these wheelchair users must rely on caretakers to assist them with their mobility, thus limiting their independence. Milpet is an effective access technology research platform centered around improving the quality of life of those confined to wheelchairs. By expanding Milpet's control interface to include speech recognition, those who cannot benefit from a joystick are given new freedoms. Utilizing a map of its environment, localization is performed using LiDAR sensor scans and a particle filtering technique. In addition to simple movement commands such as "turn left", "stop", and "go faster", the speech interface along with localization and navigation modules enable patients to navigate more complex commands. For example, commands such as "take me to the kitchen" instruct Milpet to autonomously drive to the specified location while avoiding walls and other obstacles. This self-driving wheelchair is a huge leap in improving the quality of life for the mobility impaired who cannot benefit from a joystick.

Convolution Neural Networks (CNN) are rapidly deployed in ADAS and Autonomous driving for object detection, recognition, and semantic segmentation. The prior art of supporting CNN (HW IP or multi-core SW) doesn't address efficient implementation for the first layer, YUV color space, and output stride support. The given paper proposes a new pre-processing technique to enhance CNN based HW IP or multi-core SW solution. The pre-processor enables new features namely (1) Higher parallelism for the first layer with boosting of first layer (2) Efficient YUV color space (3) Efficient output stride support. The pre-processor uses novel phase-split method to enable supporting above features. The proposed solution splits input to multiple phases based on spatial location e.g. 2 phases for YUV 4:2:0 format, 4 phases for output strides 2 etc. The proposed solution is a unified solution that enables utilization (>90%) for the first layer and reduction of bandwidth of 2-4x for output stride of 2. For YUV color space, this reduces the computation by factor 2 along saving of ∼0.1 mm2 of silicon area with negligible loss in accuracy.

Recently, vision-based Advanced Driver Assist Systems have gained broad interest. In this work, we investigate free-space detection, for which we propose to employ a Fully Convolutional Network (FCN). We show that this FCN can be trained in a selfsupervised manner and achieve similar results compared to training on manually annotated data, thereby reducing the need for large manually annotated training sets. To this end, our selfsupervised training relies on a stereo-vision disparity system, to automatically generate (weak) training labels for the color-based FCN. Additionally, our self-supervised training facilitates online training of the FCN instead of offline. Consequently, given that the applied FCN is relatively small, the free-space analysis becomes highly adaptive to any traffic scene that the vehicle encounters. We have validated our algorithm using publicly available data and on a new challenging benchmark dataset that is released with this paper. Experiments show that the online training boosts performance with 5% when compared to offline training, both for F_max and AP.