Real-time path planning for autonomous vehicle off-road driving

Real-time path planning for autonomous vehicle off-road driving Ethery Ramirez-Robles, Oleg Starostenko and Vicente Alarcon-Aquino Department of Computing, Electronics, and Mechatronics, Universidad de las Américas—Puebla, Puebla, Mexico ABSTRACT Submitted 9 February 2024 Accepted 27 June 2024 Published 24 July 2024 Corresponding author Oleg Starostenko, Academic editor Louise Dennis Additional Information and Declarations can be found on page 20 DOI 10.7717/peerj-cs.2209 Copyright 2024 Ramirez-Robles et al. Distributed under Creative Commons CC-BY 4.0 OPEN ACCESS Background. Autonomous driving is a growing research area that brings benefits in science, economy, and society. Although there are several studies in this area, currently there is no a fully autonomous vehicle, particularly, for off-road navigation. Autonomous vehicle (AV) navigation is a complex process based on application of multiple technologies and algorithms for data acquisition, management and understanding. Particularly, a self-driving assistance system supports key functionalities such as sensing and terrain perception, real time vehicle mapping and localization, path prediction and actuation, communication and safety measures, among others. Methods. In this work, an original approach for vehicle autonomous driving in off-road environments that combines semantic segmentation of video frames and subsequent real-time route planning is proposed. To check the relevance of the proposal, a modular framework for assistive driving in off-road scenarios oriented to resource-constrained devices has been designed. In the scene perception module, a deep neural network is used to segment Red-Green-Blue (RGB) images obtained from camera. The second traversability module fuses Light Detection And Ranging (LiDAR) point clouds with the results of segmentation to create a binary occupancy grid map to provide scene understanding during autonomous navigation. Finally, the last module, based on the Rapidly-exploring Random Tree (RRT) algorithm, predicts a path. The Freiburg Forest Dataset (FFD) and RELLIS-3D dataset were used to assess the performance of the proposed approach. The theoretical contributions of this article consist of the original approach for image semantic segmentation fitted to off-road driving scenarios, as well as adapting the shortest route searching A* and RRT algorithms to AV path planning. Results. The reported results are very promising and show several advantages compared to previously reported solutions. The segmentation precision achieves 85.9% for FFD and 79.5% for RELLIS-3D including the most frequent semantic classes. While compared to other approaches, the proposed approach is faster regarding computational time for path planning. Subjects Autonomous Systems, Real-Time and Embedded Systems, Neural Networks Keywords Machine vision, Semantic image segmentation, Off-road autonomous driving, Navigation path planning INTRODUCTION Autonomous vehicle (AV) navigation is quite new and still an open problem; the proposed solutions promise many benefits for the economy and society. Currently, two principal scenarios for autonomous driving are examined: on-road and off-road driving. The on-road How to cite this article Ramirez-Robles E, Starostenko O, Alarcon-Aquino V. 2024. Real-time path planning for autonomous vehicle offroad driving. PeerJ Comput. Sci. 10:e2209 http://doi.org/10.7717/peerj-cs.2209 scenario is for urban cities, when the lane markings, defined cues, speed signs, and pavement roads among other features are taken into account. Conversely, there are uneven surfaces in the off-road driving, where there are no explicit delimiters; so, vegetation, obstacles, and different terrains must be analyzed in real time for path prediction. Additionally, off-road terrains are more prone to changes due to different weather conditions, for example, there would be mud instead of dry soil and puddles can appear after the rain. In several countries, including the USA and Mexico, at least one-third of the roads are unpaved. Therefore, although there are multiple approaches for off-road vehicle navigation that in general use synthetic scene datasets, it is a challenge to improve their performance for autonomous driving assistance in a real-world unstructured environment. The research methodologies used for supporting AV navigation are subdivided into modular and end-to-end approaches. In the modular approach, there exist some subtasks to be solved independently such as AV localization, camera perception, position mapping, path planning, vehicle control, and others. In the end-to-end approach, AV assistance system looks like a black box, where the inputs are the data received from sensors and the outputs are low-level commands for the vehicle driving control. In this research, we focus on a modular approach to propose an assistive real time system oriented to work on resource-constrained devices. The principal module that defines the quality of off-road AV assistance is terrain perception and scene understanding, which are based on precise image segmentation and objects classification. One of the most promising technique is semantic segmentation used in many applications, for instance, for detecting brain tumors (Kumar, Negi & Singh, 2019; Myronenko & Hatamizadeh, 2020), recognition of road signs in urban environments (Timbus, Miclea & Lemnaru, 2018; Khan, Adhami & Bhuiyan, 2009), processing satellite images (Wurm et al., 2019; Saifi & Singla, 2020), detection of plant diseases in agriculture (Singh & Misra, 2017; Milioto, Lottes & Stachniss, 2018), and autonomous driving (Kaymak & Uçar, 2019; Treml et al., 2016), among others. It is important to highlight that during autonomous driving the resolution, perspective, and visual angle of cameras vary for different vehicles; therefore, the segmentation approach must be adjusted to particular environment. Usually, one or several sensors mounted on the AV are used for video stream gathering, such as video cameras, radiolocation system RADARs, LiDARS, GPS, and inertial measurement units (IMUs). In some cases, there is a preprocessing step that combines the data from the sensors; thus, the inputs to scene perception module are not just raw readings. Current approaches also use deep neural network (DNN) for scene perception that provide noticeable enhancing in the segmentation accuracy (Valada et al., 2017; Maturana et al., 2018). However, the suitability of DNN due to its computational cost derived from many sequential layers with tensorial operations must be evaluated, particularly, for AV driving assistance systems with limited resources and real-time computing needs. When the information about scene is obtained, the AV path planning and route prediction is accomplished. To solve this task, usually the global and local planning are provided. While in the former the goal is to find the best route considering the environment map, in the latter a vehic (...truncated)