IS&T | Library

Abstract

There are many electronic documents salient to read for each given topic; however, finding a suitable reading order for pedagogical purposes has been underserved historically by the text analytics community. In this research, we propose an automatic reading order generation technique that can suggest a suitable and optimal reading order for curriculum generation quantitatively. It is necessary to read the relevant documents in some logical order to understand the topics clearly. There are many learning pedagogies advanced, so for our purposes we use the author-supplied reading orders of salient content sets for ground truth. Our method suggests the best reading order automatically by checking the relevant topics, document distances, and semantic structure of the given documents. The system will generate a suitable and efficient reading sequence by analyzing the information, similarity, overlap of contents, and distances using word frequency, and topic sets. We measure the similarity, relevance, distance, and overlap of different documents using cosine similarity, entropy relevance, Euclidean distances, and Jaccard similarities respectively. We propose an algorithm that will generate the best possible reading order for a set of given documents. We evaluated the performance of our system against the ground truth reading order using different kinds of textbooks and generalized the finding for any given set of documents.

Digital Library: EI

Published Online: January 2024

Data Extraction, Visualization, and Storytelling: A Case Study in Headline Analysis on "The Hongkong News" with Deep Learning

44 15

data mining
data visualization
AI
machine learning
storytelling

Michael Kin-Fu Yip ; Vincent Wai-Yip Lum

DOI

10.2352/issn.2168-3204.2023.20.1.6

Volume 20

Issue 1

Abstract

This paper presents the methodologies to extract the headline and illustrations from a historical newspaper for storytelling to support digital scholarship. It explored the ways in which new digital tools can facilitate the understanding of the newspaper content in the setting of time and space, "The Hongkong News" was selected from Hong Kong Early Tabloid Newspaper for the case study owing to its uniqueness in historical value towards the scholars. The proposed methodologies were evaluated in OCR (Optical Character Recognition) with scraping and Deep Learning Object Detection models. Two visualization products were developed to showcase the feasibility of our proposed methods to serve the storytelling purpose.

Digital Library: ARCHIVING

Published Online: June 2023

Toward an Ontological Model for Audiovisual Archive Datafication

20 4

Metadata management
ontology
machine learning
digitisation
audiovisual archive

Yuchen Yang

DOI

10.2352/issn.2168-3204.2023.20.1.20

Volume 20

Issue 1

Autonomous Vehicles and Machines 2023 Conference Overview and Papers Program

Abstract

With the initiatives like Collections as Data and Computational Archival Science, archives are no longer seen as a static documentation of objects, but evolving sources of cultural and historical data. This work emphasizes the potential updates in preserving and documenting digital audiovisual (AV) content from a data perspective, considering the recent developments in natural language processing and computer vision tasks, as well as the emergence of interactive and embodied experiences and interfaces for innovatively accessing archival content. As part of Swiss national scientific fund Sinergia project, this work was able to work end-to-end with real-world AV archives like Télévision Suisse Romande (RTS). Resorting to an updated narrative model for mapping data that can be obtained from the content as well as the consumer, this work proposed an experimental attempt to build an ontology to formally sum up the potential new paradigm for preservation and accessibility from a data perspective for modern archives, in the hope for nurturing a digital and data-driven mind-set for archive practices.

Digital Library: ARCHIVING

Published Online: June 2023

Front Matter

42 30

autonomous vehicles
perception
machine learning
deep learning
sensors and processors

DOI

10.2352/EI.2023.35.16.AVM-A16

Volume 35

Issue 16

Abstract

Advancements in sensing, computing, image processing, and computer vision technologies are enabling unprecedented growth and interest in autonomous vehicles and intelligent machines, from self-driving cars to unmanned drones, to personal service robots. These new capabilities have the potential to fundamentally change the way people live, work, commute, and connect with each other, and will undoubtedly provoke entirely new applications and commercial opportunities for generations to come. The main focus of AVM is perception. This begins with sensing. While imaging continues to be an essential emphasis in all EI conferences, AVM also embraces other sensing modalities important to autonomous navigation, including radar, LiDAR, and time of flight. Realization of autonomous systems also includes purpose-built processors, e.g., ISPs, vision processors, DNN accelerators, as well core image processing and computer vision algorithms, system design and architecture, simulation, and image/video quality. AVM topics are at the intersection of these multi-disciplinary areas. AVM is the Perception Conference that bridges the imaging and vision communities, connecting the dots for the entire software and hardware stack for perception, helping people design globally optimized algorithms, processors, and systems for intelligent “eyes” for vehicles and machines.

Digital Library: EI

Published Online: January 2023

Active learning approaches to analysis of thin-film printed sensors for determining nitrate levels in soil

44 13

active learning
machine learning
image processing

Xihui Wang ; Ali Shakouri ; Bruno Ribeiro ; George T.C. Chiu ; Jan P. Allebach

DOI

10.2352/EI.2023.35.15.COLOR-194

Volume 35

Issue 15

Abstract

In order to train a learning-based prediction model, large datasets are typically required. One of the major restrictions of machine learning applications using customized databases is the cost of human labor. In the previous papers [3, 4, 5], it is demonstrated through experiments that the correlation between thin-film nitrate sensor performance and surface texture exists. In the previous papers, several methods for extracting texture features from sensor images are explored, repeated cross-validation and a hyperparameter auto-tuning method are performed, and several machine learning models are built to improve prediction accuracy. In this paper, a new way to achieve the same accuracy with a much smaller dataset of labels by using an active learning structure is presented.

Digital Library: EI

Published Online: January 2023

Machine learning estimation of camera spectral sensitivity functions with non-RGB color filters

65 17

spectral sensitivity functions
machine learning
color filter arrays
estimation

Abraham Sachs ; Ramakrishna Kakarala

DOI

10.2352/EI.2023.35.15.COLOR-199

Volume 35

Issue 15

Abstract

The spectral sensitivity functions of a digital image sensor determine the sensor’s color response to scene-radiated light. Knowing these spectral sensitivity functions is very important for applications that require accurate color, such as computer vision. Traditional measurements of these functions are time consuming, and require expensive lab equipment to generate narrow-band monochromatic light. Previous works have shown that sensitivity curves can be estimated using images of a color checker chart with known spectral reflectances, using either numerical optimization or machine learning. However, previous works in the literature have not considered sensitivity functions for CFAs (color filter arrays) other than RGB, such as RCCB (Red Clear Blue) or RYYCy (Red Yellow Cyan). Non-RGB CFAs have been shown to be useful for automotive and security camera applications, especially in low light situations. We propose a machine learning method to estimate the sensitivity curves of sensors with non-RGB filters, in addition to the RGB filters addressed previously in the literature, using a single image of a color chart under unknown illumination. Including non-RGB filters makes the estimation problem much more challenging, since the resulting space of color filters is no longer modelled by simple Gaussian shapes.

Digital Library: EI

Published Online: January 2023

Evaluating active learning for blind imbalanced domains

117 35

machine learning
deep learning
active learning
data augmentation
out of distribution
blind imbalanced domains

Hiroshi Kuwajima ; Masayuki Tanaka ; Masatoshi Okutomi

DOI

10.2352/EI.2023.35.9.IPAS-288

Volume 35

Issue 9

Imaging and Multimedia Analytics at the Edge 2023 Conference Overview and Papers Program

Abstract

Deep learning, which has been very successful in recent years, requires a large amount of data. Active learning has been widely studied and used for decades to reduce annotation costs and now attracts lots of attention in deep learning. Many real-world deep learning applications use active learning to select the informative data to be annotated. In this paper, we first investigate laboratory settings for active learning. We show significant gaps between the results from different laboratory settings and describe our practical laboratory setting that reasonably reflects the active learning use cases in real-world applications. Then, we introduce a problem setting of blind imbalanced domains. Any data set includes multiple domains, e.g., individuals in handwritten character recognition with different social attributes. Major domains have many samples, and minor domains have few samples in the training set. However, we must accurately infer both major and minor domains in the test phase. We experimentally compare different methods of active learning for blind imbalanced domains in our practical laboratory setting. We show that a simple active learning method using softmax margin and a model training method using distance-based sampling with center loss, both working in the deep feature space, perform well.

Digital Library: EI

Published Online: January 2023

Front Matter

24 20

multimedia analysis
machine learning
mobile imaging
web imaging
Internet of Things edge imaging

DOI

10.2352/EI.2023.35.7.IMAGE-A07

Volume 35

Issue 7

Abstract

Recent progress at the intersection of deep learning and imaging has created a new wave of interest in imaging and multimedia analytics topics, from social media sharing to augmented reality, from food and nutrition to health surveillance, from remote sensing and agriculture to wildlife and environment monitoring. Compared to many subjects in traditional imaging, these topics are more multi-disciplinary in nature. This conference will provide a forum for researchers and engineers from various related areas, both academic and industrial, to exchange ideas and share research results in this rapidly evolving field.

Digital Library: EI

Published Online: January 2023

Synthetic speech attribution using self supervised audio spectrogram transformer

142 49

speech forensics
synthesized speech
attribution
transformers
mel spectrograms
machine learning
deep learning
media forensics

Amit Kumar Singh Yadav ; Emily R. Bartusiak ; Kratika Bhagtani ; Edward J. Delp

DOI

10.2352/EI.2023.35.4.MWSF-372

Volume 35

Issue 4

Abstract

The ability to synthesize convincing human speech has become easier due to the availability of speech generation tools. This necessitates the development of forensics methods that can authenticate and attribute speech signals. In this paper, we examine a speech attribution task, which identifies the origin of a speech signal. Our proposed method known as Synthetic Speech Attribution Transformer (SSAT) converts speech signals into mel spectrograms and uses a self-supervised pretrained transformer for attribution. This transformer is pretrained on two large publicly available audio datasets: Audio Set and LibriSpeech. We finetune the pretrained transformer on three speech attribution datasets: the DARPA SemaFor Audio Attribution dataset, the ASVspoof2019 dataset, and the 2022 IEEE SP Cup dataset. SSAT achieves high closed-set accuracy on all datasets (99.8% on ASVspoof2019 dataset, 96.3% on SP Cup dataset, and 93.4% on DARPA SemaFor Audio Attribution dataset). We also investigate the methodâ€™s ability to generalize to unknown speech generation methods (open-set scenario). SSAT has high performance, achieving an open-set accuracy of 90.2% on the ASVspoof2019 dataset and 88.45% on DARPA SemaFor Audio Attribution dataset. Finally, we show that our approach is robust to typical compression rates used by YouTube for speech signals.

Digital Library: EI

Published Online: January 2023

LECA: A learned approach for efficient cover-agnostic watermarking

159 41

watermarking
deep learning
machine learning

Xiyang Luo ; Michael Goebel ; Elnaz Barshan ; Feng Yang

DOI

10.2352/EI.2023.35.4.MWSF-376

Volume 35

Issue 4