Tiny-YOLO distance measurement and object detection coordination system for the BarelangFC robot

International Journal of Electrical and Computer Engineering (IJECE) Vol. 13, No. 6, December 2023, pp. 6926~6939 ISSN: 2088-8708, DOI: 10.11591/ijece.v13i6.pp6926-6939  6926 Tiny-YOLO distance measurement and object detection coordination system for the BarelangFC robot Susanto, Jony Arif Ricardo Silitonga, Riska Analia, Eko Rudiawan Jamzuri, Daniel Sutopo Pamungkas Department of Electrical Engineering, Politeknik Negeri Batam, Batam, Indonesia Article Info ABSTRACT Article history: A humanoid robot called BarelangFC was designed to take part in the Kontes Robot Indonesia (KRI) competition, in the robot coordination division. In this division, each robot is expected to recognize its opponents and to pass the ball towards a team member to establish coordination between the robots. In order to achieve this team coordination, a fast and accurate system is needed to detect and estimate the other robot’s position in real time. Moreover, each robot has to estimate its team members’ locations based on its camera reading, so that the ball can be passed without error. This research proposes a Tiny-YOLO deep learning method to detect the location of a team member robot and presents a real-time coordination system using a ZED camera. To establish the coordinate system, the distance between the robots was estimated using a trigonometric equation to ensure that the robot was able to pass the ball towards another robot. To verify our method, real-time experiments was carried out using an NVDIA Jetson NX Xavier, and the results showed that the robot could estimate the distance correctly before passing the ball toward another robot. Received Mar 3, 2023 Revised May 23, 2023 Accepted Jun 4, 2023 Keywords: Deep learning Humanoid robot NVIDIA Jetson NX Xavier Real-time Robot coordination This is an open access article under the CC BY-SA license. Corresponding Author: Susanto Department of Electrical Engineering, Politeknik Negeri Batam Ahmad Yani street, Batam Center, Batam, Kepulauan Riau, 29461, Indonesia Email: 1. INTRODUCTION Humanoid robots, i.e., robots resembling human beings, have developed rapidly in recent years. Many researchers have developed these kinds of robots for various purposes, such as learning assistance for primary education [1], clinical applications [2], [3], playing games [4], [5], assisting the elderly [6], and even playing soccer. Humanoid robots have been developed to play football in the same way as human beings, such as passing the ball to a nearby player, kicking the ball towards the goal, and recognizing other team members on the field in real time. In order to allow a robot to play football on a field, we need to consider the robot's vision and coordination strategies. Numerous researchers have proposed several methods of achieving a vision system for detecting an object. As reported previously, we can forward a modest object detection system using CVblobs and the Hough circle method (CBHM) to detect a white ball on the field. Another method for detecting objects was introduced in [7], [8]; Maiettini et al. [7] used a convolutional neural network (CNN) to train a network in an end-to-end manner on larger datasets with 2D bounding objects, while in [8], a CNN was used to predict the class of an object from the proposed region. Aslan et al. [9] introduced semantic segmentation algorithms to a simulation and compared the accuracy, segmentation performance, and number of parameters. A year later, they combined a semantic algorithm with deep reinforcement learning (DRL) to recognize an object moving toward the robot [10]. Journal homepage: http://ijece.iaescore.com Int J Elec & Comp Eng ISSN: 2088-8708  6927 In the domain of vision detection algorithms, some studies have used a fast and accurate detection system called you only look once (YOLO). This method can process images for real-time applications at a rate of 45 frames per second (FPS). The YOLO algorithm was later extended to YOLOv2, which could predict object classes without the need for labeled detection data. Redmon and Farhadi [11] then introduced YOLOv3, and presented results that were more accurate from a model that was three times faster than solid state drive (SSD). Another object detection method called XNOR-Networks could estimate the convolutions using a primary binary operation. In our previous work, we adapted this network and combined it with YOLOv3 to detect the ball and goal, using a model where the layer configuration resembled the Tiny-YOLO and the model was run on an NVIDIA Jetson TX1 [12]. Another important aspect of developing a humanoid robot for soccer playing is robot localization. This helps the robot to move automatically across the field to get the ball, pass it to other team members, and kick it towards the goal. In recent years, many methods of robot localization have been developed, and particularly for humanoid robots. Fourmy et al. [13] proposed a visual-inertial navigation system to localize a robot in a 3D indoor environment by employing sensors such as inertial measurement unit (IMU), coders, vision, and/or light detection and ranging (LiDAR). Another popular localization algorithm is simultaneous localization and mapping (SLAM), which has been implemented in different ways; for example, Raghavan et al. [14] combined the state-of-art odometry with mapping based on LiDAR data and inertial kinematics, while Zhang et al. [15] implemented a graph-based segmentation from RGB-D point clouds to achieve robust, dense RGB-D environment reconstruction. In [16], an RGB-D camera was combined with a depth descriptor to track features even in a sequence with seriously blurred images, and the scheme in [17] used odometry data acquired from the fusion of visual and robot odometry information. A Kinect sensor can also be used for robot localization, as described in [18], in which a depth map was used to extract the location and a global planning algorithm was applied to understand the surrounding environment. In addition to localization, a fuzzy Markov decision process (FMDP) can also be used for path planning for robot localization [19]. Moreover, Monte Carlo localization (MCL) can be considered in order to achieve robot localization. Hartfill [20] developed an MCL scheme based on a 2D RGB image to retrieve the localization information and evaluate it in simulation. Dalmasso et al. [21] used a Monte Carlo tree search (MCTS) to decentralize one robot with human to understand the location. In recent research on robot localization, the robot’s self-position on a global map has been estimated based on its sensor measurements. However, self-localization is not sufficient for a typical robot soccer competition, as team coordination is required. In addition to estimating its self-position, the robot must estimate the locations of the ball, the goal, and the other robots on the global map. In addition, the position estimation of the object must be carried out using a se (...truncated)