IS&T | Library

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

Abstract

Automated extraction of intersection topologies from aerial and street-level images is relevant for Smart City traffic-control and safety applications. The intersection topology is expressed in the amount of approach lanes, the crossing (conflict) area, and the availability of painted striping for guidance and road delineation. Segmentation of road surface and other basic information can be obtained with 80% score or higher, but the segmentation and modeling of intersections is much more complex, due to multiple lanes in various directions and occlusion of the painted stripings. This paper addresses this complicated problem by proposing a dualistic channel model featuring direct segmentation and involving domain knowledge. These channels are developing specific features such as drive lines and lane information based on painted striping, which are filtered and then fused to determine an intersection-topology model. The algorithms and models are evaluated with two datasets, a large mixture of highway and urban intersections and a smaller dataset with intersections only. Experiments with measuring the GEO metric show that the proposed late-fusion system increases the recall score with 47 percentage points. This recall gain is consistent for using either aerial imagery or a mixture of aerial and street-level orthographic image data. The obtained recall for intersections is much lower than for highway data because of the complexity, occlusions by trees and the small amount of annotated intersections. Future work should aim at consolidating this model improvement at a higher recall level with more annotated data on intersections.

Digital Library: EI

Published Online: January 2024

Proceedings

144 12

Deep Learning
Distributed Training
High Performance Computing
Segmentation

Isaac Lyngaas, Murali Gopalakrishnan Meena, Evan Calabrese, Mohamed Wahib, Peng Chen, Jun Igarashi, Yuankai Huo, Xiao Wang

DOI

10.2352/EI.2024.36.12.HPCI-199

Volume 36

Issue 12

Segmentation of Starch Granules in Microscopic Images Using a U-Net Model

Abstract

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Digital Library: EI

Published Online: January 2024

Proceedings

32 4

Quantitative analysis
Segmentation
Deep learning
Evaluation metrics
Starch U-net

Ye Jin, Pierce Cui, Jinshan Tang

DOI

10.2352/EI.2024.36.5.MLSI-310

Volume 36

Issue 5

Abstract

Starch plays a pivotal role in human society, serving as a vital component of our food sources and finding widespread applications in various industries. Microscopic imaging offers a straightforward, efficient, and precise approach to examine the distribution, morphology, and dimensions of starch granules. Quantitative analysis through the segmentation of starch granules from the background aids researchers in exploring their physicochemical properties. This article presents a novel approach utilizing a modified U-Net model in deep learning to achieve the segmentation of starch granule microscope images with remarkable accuracy. The method yields impressive results, with mean values for several evaluation metrics including JS, Dice, Accuracy, Precision, Sensitivity and Specificityreaching 89.67%, 94.55%, 99.40%, 94.89%, 94.23% and 99.70%, respectively.

Digital Library: EI

Published Online: January 2024

Reflectance Transformation Imaging (RTI) Data Analysis for Change Detection: Application to Monitoring Protective Coating Failure on Low Carbon Steel

46 9

Reflectance Transformation Imaging
Coating failure
Segmentation
Features analysis

Amalia Siatou, Marvin Nurit, Sunita Saha, Gaëtan Le Goïc, Laura Brambilla, Christian Degrigny, Robert Sitnik, Alamin Mansouri

Pages 42 - 47, June 2023, This work is licensed under the Creative Commons Attribution 4.0 International License. 2023

DOI

10.2352/issn.2168-3204.2023.20.1.8

Volume 20

Issue 1

Abstract

This paper examines two new methodological approaches exploring Reflectance Transformation Imaging (RTI) data processing for detecting, documenting, and tracking surface changes. The first approach is unsupervised and applies per-pixel calculations on the raw image stack to extract information related to specific surface attributes (angular reflectance, micro-geometry). The second method proposes a supervised segmentation approach that, based on machine learning algorithms, uses coefficients of a fitting model to separate the surface’s characteristics and assign them to a class. Both methodologies were applied to monitor coating failure, in the form of filiform corrosion, on low carbon steel test samples, mimicking treated historical metal objects’ surfaces. The results demonstrate the feasibility of creating accurate cartographies that depict the surface characteristics and their location. Additionally, they provide a qualitative evaluation of corrosion progression that allows tracking and monitoring changes on challenging surfaces.

Digital Library: ARCHIVING

Published Online: June 2023

Few-shot learning on point clouds for railroad segmentation

158 72

Few Shot Learning
Point Cloud
Segmentation
Railroad

Abdur Razzaq Fayjie, Patrick Vandewalle

DOI

10.2352/EI.2023.35.17.3DIA-100

Volume 35

Issue 17

Abstract

Infrastructure maintenance of complex environments like railroads is a very expensive operation. Recent advances in mobile mapping systems to collect 3D point cloud data and in deep learning for detection and segmentation can prove to be very helpful in automating this maintenance and allowing preventive maintenance at certain locations before big failures occur. Some fully-supervised methods have been developed for understanding dynamic railroad environments. These methods often fail to generalize to infrastructure changes or new classes in low-labeled data. To address this issue, we propose a railroad segmentation method that leverages few-shot learning by generating class prototypes for the most relevant infrastructure classes. This method takes advantage of existing embedding networks for point clouds, taking the geometrical and spatial context into account for feature representation of complex connected classes. We evaluate our method on real-world data measured on Belgian railway tracks. Our model achieves promising results on connected classes, exposed to only a few annotated samples at test time.

Digital Library: EI

Published Online: January 2023

Physics guided machine learning for multi-material decomposition of tissues from dual-energy CT scans of simulated breast models with calcifications

134 52

Material decomposition
Unsupervised learning
Deep Learning
Computed tomography
High Performance Computing
Segmentation

Muralikrishnan Gopalakrishnan Meena, Amir K. Ziabari, Singanallur V. Venkatakrishnan, Isaac R. Lyngaas, Matthew R. Norman, Balint Joo, Thomas L. Beck, Charles A. Bouman, Anuj J. Kapadia, Xiao Wang

DOI

10.2352/EI.2023.35.11.HPCI-228

Volume 35

Issue 11

Abstract

We introduce a physics guided data-driven method for image-based multi-material decomposition for dual-energy computed tomography (CT) scans. The method is demonstrated for CT scans of virtual human phantoms containing more than two types of tissues. The method is a physics-driven supervised learning technique. We take advantage of the mass attenuation coefficient of dense materials compared to that of muscle tissues to perform a preliminary extraction of the dense material from the images using unsupervised methods. We then perform supervised deep learning on the images processed by the extracted dense material to obtain the final multi-material tissue map. The method is demonstrated on simulated breast models with calcifications as the dense material placed amongst the muscle tissues. The physics-guided machine learning method accurately decomposes the various tissues from input images, achieving a normalized root-mean-squared error of 2.75%.

Digital Library: EI

Published Online: January 2023

Terrain segmentation for commercial vehicles and working machines

64 26

Scene analysis for intelligent robots
Field robotics
Segmentation
Terrain classification

Raimund Edlinger, Ulrich Mitterhuber, Andreas Nüchter

DOI

10.2352/EI.2023.35.5.IRIACV-324

Volume 35

Issue 5

Improving an inkjet printer: saturation enhancement based on segmentation and hue

Abstract

In the field of automated working machines, not only is the general trend towards automation in industry, transport and logistics reflected, but new areas of application and markets are also constantly emerging. In this paper we present a pipeline for terrain classification in offroad environments and in the field of "automated maintenance of slopes", which offers potential for solving numerous socio-economic needs. Working tasks can be made more efficient, more ergonomic and, in particular, much safer, because mature, automated vehicles are used. At present, however, such tasks can only be carried out remotely or semi-automatically, under the supervision of a trained specialist. This only partially facilitates the work. The real benefit only comes when the supervising person is released from this task and is able to pursue other work. In addition to the development of a safe integrated system and sensor concept for use in public spaces as a basic prerequisite for vehicles licensed in the future, increased situational awareness of mobile systems through machine learning in order to increase their efficiency and flexibility is also of great importance.

Digital Library: EI

Published Online: January 2023

Articles

24 2

Saturation enhancement
Segmentation
Hue

Sige Hu, Baekdu Choi, George Chiu, Zillion Lin, Davi He, Jan Allebach

DOI

10.2352/ISSN.2470-1173.2021.16.COLOR-328

Volume 33

Issue 16

Sky Segmentation for Enhanced Depth Reconstruction and Bokeh Rendering with Efficient Architectures

In one of our previous paper [1] proposed in the last year, we described the color management pipeline that applied to our nail inkjet printer. However, the resulting prints are not as vivid as we would like to have since those prints are not well saturated. In this paper, we propose a saturation enhancement method based on the image segmentation and hue angle. This method will not necessarily give us the closest representation of the colors within the input image but could give us more saturated prints. The main idea that we perform our saturation enhancement method is to keep the lightness and hue constant, while stretching the chroma component.

Digital Library: EI

Published Online: January 2021

Articles

39 4

Deep Learning
Computer Vision
Segmentation
Depth
Bokeh
Sky
Neural Network

Tyler Nuanes, Matt Elsey, Radek Grzeszczuk, John Paul Shen

DOI

10.2352/ISSN.2470-1173.2020.14.COIMG-378

Volume 32

Issue 14

Detection and Characterization of Rumble Strips in Roadway Video Logs

We present a high-quality sky segmentation model for depth refinement and investigate residual architecture performance to inform optimally shrinking the network. We describe a model that runs in near real-time on mobile device, present a new, highquality dataset, and detail a unique weighing to trade off false positives and false negatives in binary classifiers. We show how the optimizations improve bokeh rendering by correcting stereo depth misprediction in sky regions. We detail techniques used to preserve edges, reject false positives, and ensure generalization to the diversity of sky scenes. Finally, we present a compact model and compare performance of four popular residual architectures (ShuffleNet, MobileNetV2, Resnet-101, and Resnet-34-like) at constant computational cost.

Digital Library: EI

Published Online: January 2020

Articles

36 3

Computer vision
Autonomy
Transportation Systems
Deep Learning
Segmentation

Deniz Aykac, Thomas Karnowski, Regina Ferrell, James S. Goddard

DOI

10.2352/ISSN.2470-1173.2020.6.IRIACV-050

Volume 32

Issue 6