IS&T | Library

Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model

54 4

Person Re-Identification
Scene Understanding
Spatio-Temporal Model
Multi-Camera Network
Multiple Granularity Network

Xin Liu, Herman G.J. Groot, Egor Bondarev, Peter H.N. de With

Pages 95-1 - 95-13, January 2020, © Society for Imaging Science and Technology 2020

DOI

10.2352/ISSN.2470-1173.2020.10.IPAS-095

Volume 32

Issue 10

In this paper, we investigate person re-identification (re-ID) in a multi-camera network for surveillance applications. To this end, we create a Spatio-Temporal Multi-Camera model (ST-MC model), which exploits statistical data on a person’s entry/exit points in the multi-camera network, to predict in which camera view a person will re-appear. The created ST-MC model is used as a novel extension to the Multiple Granularity Network (MGN) [1], which is the current state of the art in person re-ID. Compared to existing approaches that are solely based on Convolutional Neural Networks (CNNs), our approach helps to improve the re-ID performance by considering not only appearance-based features of a person from a CNN, but also contextual information. The latter serves as scene understanding information complimentary to person re-ID. Experimental results show that for the DukeMTMC-reID dataset [2][3], introduction of our ST-MC model substantially increases the mean Average Precision (mAP) and Rank-1 score from 77.2% to 84.1%, and from 88.6% to 96.2%, respectively.

Digital Library: EI

Published Online: January 2020

Automatic shadow detection using hyperspectral data for terrain classification

133 5

Hyperspectral vision
Scene Understanding
Autonomous driving
Computer vision
Machine learning
Shadow detection
Image processing

Christian Winkens, Veronika Adams, Dietrich Paulus

Pages 31-1 - 31-6, January 2019, © Society for Imaging Science and Technology 2019

DOI

10.2352/ISSN.2470-1173.2019.15.AVM-031

Volume 31

Issue 15

Hyperspectral image classification has received more attention from researchers in recent years. Hyperspectral imaging systems utilize sensors, which acquire data mostly from the visible through the near infrared wavelength ranges and capture tens up to hundreds of spectral bands. Using the detailed spectral information, the possibility of accurately classifying materials is increased. Unfortunately conventional spectral cameras sensors use spatial or spectral scanning during acquisition which is only suitable for static scenes like earth observation. In dynamic scenarios, such as in autonomous driving applications, the acquisition of the entire hyperspectral cube in one step is mandatory. To allow hyperspectral classification and enhance terrain drivability analysis for autonomous driving we investigate the eligibility of novel mosaic-snapshot based hyperspectral cameras. These cameras capture an entire hyperspectral cube without requiring moving parts or line-scanning. The sensor is mounted on a vehicle in a driving scenario in rough terrain with dynamic scenes. The captured hyperspectral data is used for terrain classification utilizing machine learning techniques. A major problem, however, is the presence of shadows in captured scenes, which degrades the classification results. We present and test methods to automatically detect shadows by taking advantage of the near-infrared (NIR) part of spectrum to build shadow maps. By utilizing these shadow maps a classifier may be able to produce better results and avoid misclassifications due to shadows. The approaches are tested on our new hand-labeled hyperspectral dataset, acquired by driving through suburban areas, with several hyperspectral snapshotmosaic cameras.

Digital Library: EI

Published Online: January 2019

Deep dimension reduction for spatial-spectral road scene classification

39 8

Hyperspectral Vision
Deep Learning
Computer Vision
Machine Learning
Scene Understanding
Autonomous Driving

Christian Winkens, Florian Sattler, Dietrich Paulus

Pages 49-1 - 49-9, January 2019, © Society for Imaging Science and Technology 2019

DOI

10.2352/ISSN.2470-1173.2019.15.AVM-049

Volume 31

Issue 15

Semantic segmentation is an essential aspect of modern autonomous driving systems, since a precise understanding of the environment is crucial for navigation. We investigate the eligibility of novel snapshot hyperspectral cameras ?which capture a whole spectrum in one shot? for road scene classification. Hyperspectral data brings an advantage, as it allows a better analysis of the material properties of objects in the scene. Unfortunately, most classifiers suffer from the Hughes effect when dealing with high-dimensional hyperspectral data. Therefore we propose a new framework of hyperspectral-based feature extraction and classification. The framework utilizes a deep autoencoder network with additional regularization terms which focus on the modeling of latent space rather than the reconstruction error to learn a new dimension-reduced representation. This new dimension-reduced spectral feature space allows the use of deep learning architectures already established on RGB datasets.

Digital Library: EI

Published Online: January 2019

Context Aware Hyperspectral Scene Analysis

41 10

Hyperspectral vision
Computer vision
Machine learning
Image processing
Scene Understanding
Autonomous driving

Christian Winkens, Dietrich Paulus

DOI

10.2352/ISSN.2470-1173.2018.17.AVM-346

Volume 30

Issue 17

Hyperspectral imaging increases the amount of information incorporated per pixel in comparison to normal color cameras. Conventional hyperspectral sensors as used in satellite imaging utilize spatial or spectral scanning during acquisition which is only suitable for static scenes. In dynamic scenarios, such as in autonomous driving applications, the acquisition of the entire hyperspectral cube at the same time is mandatory. In this work, we investigate the eligibility of novel snapshot hyperspectral cameras in dynamic scenarios such as in autonomous driving applications. These new sensors capture a hyperspectral cube containing 16 or 25 spectra without requiring moving parts or line-scanning. These sensors were mounted on land vehicles and used in several driving scenarios in rough terrain and dynamic scenes. We captured several hundred gigabytes of hyperspectral data which were used for terrain classification. We propose a random-forest classifier based on hyperspectral and spatial features combined with fully connected conditional random fields ensuring local consistency and context aware semantic scene segmentation. The classification is evaluated against a novel hyperspectral ground truth dataset specifically created for this purpose.

Digital Library: EI

Published Online: January 2018

RECfusion: Automatic Scene Clustering and Tracking in Videos from Multiple Sources

106 0

Data Mining
Scene Understanding
Computer Vision
Prosumer Multimedia Stream
Cloud-based Application
Mobile Ultra-broadband Network

Filippo L.M Milotta, Sebastiano Battiato, Filippo Stanco, Valeria D’Amico, Giovanni Torrisi, Luca Addesso

DOI

10.2352/ISSN.2470-1173.2016.7.MOBMU-292

Volume 28

Issue 7