IS&T | Library

VotingNet: Adaptive Hough Voting Based Compositional Model for X-ray Prohibited Item Detection Under Occlusion

Abstract

Typing remains a primary mode of interaction in extended reality (XR), yet most XR systems still depend on physical keyboards or specialized sensing hardware. We present a visiononly keystroke tracking pipeline for surface typing that combines visible-key detection, recovery of hand-occluded keys, and finger keypress event detection on monocular video. The system first detects the keyboard and visible keys with a YOLO-based detector trained on oriented key boxes and synthetic hand-occlusion augmentation. Occluded or missed keys are then recovered with lightweight per-key regressors driven by an efficient 3–2 recovery-key selection strategy. For keypress detection, we use monocular 3D hand reconstruction with HaMeR and an autoregressive transformer that processes temporal image sequences, recovered hand meshes, finger keypoint masks, and camera poses to estimate press-down probabilities for each finger. On the Keyboard Key Detection dataset, our visible-key detector achieves 97% mAP@75 with mean IoU around 0.9, while occluded-key recovery sustains 30 fps. On typing videos from the MSU Typing Behavior Database, the keypress module reaches up to 90% event detection accuracy with 28 ms latency. Together, these components form a practical end-to-end vision system for hardware-free keystroke tracking on printed, projected, or rendered keyboard layouts.

Digital Library: EI

Published Online: March 2026

Proceedings Paper

73 16

Object detection
Security inspection
Occlusion
X-ray images
Compositional model
Voting

Kaitao Huang, Yan Yan

DOI

10.2352/EI.2025.37.8.IMAGE-270

Volume 37

Issue 8

Crowd counting using deep learning based head detection

Abstract

Recently, X-ray prohibited item detection has been widely used for security inspection. In practical applications, the items in the luggage are severely overlapped, leading to the problem of occlusion. In this paper, we address prohibited item detection under occlusion from the perspective of the compositional model. To this end, we propose a novel VotingNet for occluded prohibited item detection. VotingNet incorporates an Adaptive Hough Voting Module (AHVM) based on the generalized Hough transform into the widely-used detector. AHVM consists of an Attention Block (AB) and a Voting Block (VB). AB divides the voting area into multiple regions and leverages an extended Convolutional Block Attention Module (CBAM) to learn adaptive weights for inter-region features and intra-region features. In this way, the information from unoccluded areas of the prohibited items is fully exploited. VB collects votes from the feature maps of different regions given by AB. To improve the performance in the presence of occlusion, we combine AHVM with the original convolutional branches, taking full advantage of the robustness of the compositional model and the powerful representation capability of convolution. Experimental results on OPIXray and PIDray datasets show the superiority of VotingNet on widely used detectors (including representative anchor-based and anchor-free detectors).

Digital Library: EI

Published Online: February 2025

Article

442 135

Object detection
Convolutional Neural Networks
Deep Learning
YOLO
Yolov5
Precision
Mean average Precision

Maryam Hassan, Farhan Hussain, Sultan Daud Khan, Mohib Ullah, Mudassar Yamin, Habib Ullah

Pages 293--1 - 293-6, January 2023, This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. 2023

DOI

10.2352/EI.2023.35.9.IPAS-293

Volume 35

Issue 9

Abstract

Scale invariance and high miss detection rates for small objects are some of the challenging issues for object detection and often lead to inaccurate results. This research aims to provide an accurate detection model for crowd counting by focusing on human head detection from natural scenes acquired from publicly available datasets of Casablanca, Hollywood-Heads and Scut-head. In this study, we tuned a yolov5, a deep convolutional neural network (CNN) based object detection architecture, and then evaluated the model using mean average precision (mAP) score, precision, and recall. The transfer learning approach is used for fine-tuning the architecture. Training on one dataset and testing the model on another leads to inaccurate results due to different types of heads in different datasets. Another main contribution of our research is combining the three datasets into a single dataset, including every kind of head that is medium, large and small. From the experimental results, it can be seen that this yolov5 architecture showed significant improvements in small head detections in crowded scenes as compared to the other baseline approaches, such as the Faster R-CNN and VGG-16-based SSD MultiBox Detector.

Digital Library: EI

Published Online: January 2023

Boosting computer vision performance by enhancing camera ISP

219 54

Image signal processors (ISP)
Computer vision
Deep learning
Convolutional neural networks (CNN)
Object detection
Face recognition
Stereo disparity estimation

Peter van Beek, Chyuan-Tyng (Roger) Wu, Baishali Chaudhury, Thomas R. Gardos

Pages 174-1 - 174-8, January 2021, © Society for Imaging Science and Technology 2021

DOI

10.2352/ISSN.2470-1173.2021.17.AVM-174

Volume 33

Issue 17

Traditional image signal processors (ISPs) are primarily designed and optimized to improve the image quality perceived by humans. However, optimal perceptual image quality does not always translate into optimal performance for computer vision applications. In [1], Wu et al. proposed a set of methods, termed VisionISP, to enhance and optimize the ISP for computer vision purposes. The blocks in VisionISP are simple, content-aware, and trainable using existing machine learning methods. VisionISP significantly reduces the data transmission and power consumption requirements by reducing image bit-depth and resolution, while mitigating the loss of relevant information. In this paper, we show that VisionISP boosts the performance of subsequent computer vision algorithms in the context of multiple tasks, including object detection, face recognition, and stereo disparity estimation. The results demonstrate the benefits of VisionISP for a variety of computer vision applications, CNN model sizes, and benchmark datasets.

Digital Library: EI

Published Online: January 2021

Automatic Annotation of American Football Video Footage for Game Strategy Analysis

252 38

American football
Artificial intelligence
Deep learning
Object detection
Player labeling
Sports strategy analysis

Jacob Newman, Jian-Wei Lin, Dah-Jye Lee, Jen-Jui Liu

DOI

10.2352/ISSN.2470-1173.2021.6.IRIACV-303

Volume 33

Issue 6

Annotation and analysis of sports videos is a challenging task that, once accomplished, could provide various benefits to coaches, players, and spectators. In particular, American Football could benefit from such a system to provide assistance in statistics and game strategy analysis. Manual analysis of recorded American football game videos is a tedious and inefficient process. In this paper, as a first step to further our research for this unique application, we focus on locating and labeling individual football players from a single overhead image of a football play immediately before the play begins. A pre-trained deep learning network is used to detect and locate the players in the image. A ResNet is used to label the individual players based on their corresponding player position or formation. Our player detection and labeling algorithms obtain greater than 90% accuracy, especially for those skill positions on offense (Quarterback, Running Back, and Wide Receiver) and defense (Cornerback and Safety). Results from our preliminary studies on player detection, localization, and labeling prove the feasibility of building a complete American football strategy analysis system using artificial intelligence.

Digital Library: EI

Published Online: January 2021

LiDAR-Camera Fusion for 3D Object Detection

264 98

Object detection
Sensor Fusion
Robotics
Computer Vision
LiDAR
camera
3d

Darshan Bhanushali, Robert Relyea, Karan Manghi, Abhishek Vashist, Clark Hochgraf, Amlan Ganguly, Andres Kwasinski, Michael E. Kuhl, Raymond Ptucha

DOI

10.2352/ISSN.2470-1173.2020.16.AVM-257

Volume 32

Issue 16

The performance of autonomous agents in both commercial and consumer applications increases along with their situational awareness. Tasks such as obstacle avoidance, agent to agent interaction, and path planning are directly dependent upon their ability to convert sensor readings into scene understanding. Central to this is the ability to detect and recognize objects. Many object detection methodologies operate on a single modality such as vision or LiDAR. Camera-based object detection models benefit from an abundance of feature-rich information for classifying different types of objects. LiDAR-based object detection models use sparse point clouds, where each point contains accurate 3D position of object surfaces. Camera-based methods lack accurate object to lens distance measurements, while LiDAR-based methods lack dense feature-rich details. By utilizing information from both camera and LiDAR sensors, advanced object detection and identification is possible. In this work, we introduce a deep learning framework for fusing these modalities and produce a robust real-time 3D bounding box object detection network. We demonstrate qualitative and quantitative analysis of the proposed fusion model on the popular KITTI dataset.

Digital Library: EI

Published Online: January 2020

Signal detection theory and automotive imaging

237 66

image quality
Detection probability
Object detection
Automotive imaging
Noise equivalent quanta
Ideal Observer

Paul J. Kane

DOI

10.2352/ISSN.2470-1173.2019.15.AVM-027

Volume 31

Issue 15