Back to articles
Volume: 32 | Article ID: art00005
Locating Mechanical Switches Using RGB-D Sensor Mounted on a Disaster Response Robot
  DOI :  10.2352/ISSN.2470-1173.2020.6.IRIACV-016  Published OnlineJanuary 2020

To achieve one of the tasks required for disaster response robots, this paper proposes a method for locating 3D structured switches’ points to be pressed by the robot in disaster sites using RGBD images acquired by Kinect sensor attached to our disaster response robot. Our method consists of the following five steps: 1)Obtain RGB and depth images using an RGB-D sensor. 2) Detect the bounding box of switch area from the RGB image using YOLOv3. 3)Generate 3D point cloud data of the target switch by combining the bounding box and the depth image.4)Detect the center position of the switch button from the RGB image in the bounding box using Convolutional Neural Network (CNN). 5)Estimate the center of the button’s face in real space from the detection result in step 4) and the 3D point cloud data generated in step3) In the experiment, the proposed method is applied to two types of 3D structured switch boxes to evaluate the effectiveness. The results show that our proposed method can locate the switch button accurately enough for the robot operation.

Subject Areas :
Views 105
Downloads 1
 articleview.views 105
 articleview.downloads 1
  Cite this article 

Takuya Kanda, Kazuya Miyakawa, Jeonghwang Hayashi, Jun Ohya, Hiroyuki Ogata, Kenji Hashimoto, Xiao Sun, Takashi Matsuzawa, Hiroshi Naito, Atsuo Takanishi, "Locating Mechanical Switches Using RGB-D Sensor Mounted on a Disaster Response Robotin Proc. IS&T Int’l. Symp. on Electronic Imaging: Intelligent Robotics and Industrial Applications using Computer Vision,  2020,  pp 16-1 - 16-7,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2020
Electronic Imaging
Society for Imaging Science and Technology
7003 Kilworth Lane, Springfield, VA 22151 USA