Testing driver warning systems for off-road industrial vehicles using a cyber-physical simulator

Journal on Multimodal User Interfaces https://doi.org/10.1007/s12193-024-00435-y ORIGINAL PAPER Testing driver warning systems for off-road industrial vehicles using a cyber-physical simulator Dan Garcia-Carrillo1 · Roberto Garcia1 Victor Corcoba1 · Sara Paiva2 · Xabiel G. Pañeda1 · Filipa Mourao2 · David Melendi1 · Received: 10 January 2024 / Accepted: 14 May 2024 © The Author(s) 2024 Abstract ADAS (Advanced Driver Assistance Systems) are becoming increasingly popular in on-road vehicles due to their safety, productivity, and cost savings. In the same way, off-road vehicles can benefit from ADAS systems to improve the security of drivers and workers in industrial settings. In this work, we study, design, and develop a novel security system to be integrated into industrial vehicles. This system is built to provide one-way Human Computer Interaction, from the computer to the human, so providing, through the interaction with the ADAS system, feedback to drivers about their surroundings, such as nearby workers, and thus helping to avoid collisions and prevent incidents. The study evaluates the quality of different feedback mechanisms, with the goal of designing the ADAS that produces the best User eXperience (UX). These feedback mechanisms are generated by LEDs in different display formats and colors, as well as with haptic feedback. We created a hybrid testbed using a realistic ADAS and a forklift simulator, integrating the system into a physical structure that resembles an industrial vehicle (a forklift) and used a computer-based simulation of a warehouse to gather the information from users. We performed a study with 36 people for the evaluation of the different feedback mechanisms tested, evaluating the results both from an objective point of view based on the results of the simulation, and a subjective point of view through a questionnaire and the stress of the users in each test. Keywords ADAS · HCI · Industrial vehicles · Security · Human-ADAS interaction 1 Introduction B Dan Garcia-Carrillo Roberto Garcia Xabiel G. Pañeda Filipa Mourao David Melendi Victor Corcoba Sara Paiva 1 Department of Computer Science, University of Oviedo, Campus de Viesques, 33204 Gijón, Asturias, Spain 2 ADiT-Lab, Instituto Politécnico de Viana do Castelo, Rua Escola Industrial e Comercial de Nun’Álvares, 4900-367 Viana do Castelo, Portugal Driving in industrial environments and operating large offroad vehicles are challenging tasks. Most of the vehicles are large and heavy, having moveable and articulated parts that might seriously damage other vehicles or the building structure. Even more serious, in the case that a worker is directly hit, there is a good chance that the harm caused will be severe and may even be fatal. Additionally, the environments in which these machines are used may be low visibility environments, such as dimly lit indoor environments, having to operate at night, with adverse weather conditions, or with the presence of suspended dust. The low visibility can even be caused by the actual day-today tasks, such as having to handle the load. Furthermore, heavy machines that move swiftly are utilized in different situations, such as logistical activities within an industrial facility, where they move with the presence of employees on 123 Journal on Multimodal User Interfaces foot throughout the industrial facility. It can often be challenging to detect moving impediments because of ambient noise, poor illumination, or the design of the workplace. All these situations call for a high level of skill and mental effort from the professionals involved. Accident reduction in industrial contexts is now essential in the context of the twenty-first-century factory [1]. For example, in the year 2022, 108 accidents involving a forklift were recorded in the United States of America [2]. Additionally, there are studies that indicate that a forklift is involved in one of every six serious work accidents in the United States and that pedestrians are also involved in 80% of accidents [3]. As a result, accidents caused by industrial vehicles while moving are one of the most significant dangers in all types of workplaces such as logistical centers. For these reasons, in the upcoming years, reducing or, if possible, eliminating accidents will be a first-level technology challenge in the connected industry. To address these challenges, we look at the advances in security in other related sectors, such as on-road vehicles. The use of ADAS (Advanced Driver Assis tance Systems) is increasing and, in some cases, is even mandatory. For this reason, we explore the use of ADAS in the context of off-road vehicles to advance towards reducing accidents in industrial environments. An ADAS is a set of electronic technologies designed to assist drivers in operating their vehicles more safely and efficiently. These systems use sensors, cameras, and other technologies to monitor the vehicle’s surroundings and provide real-time information to the driver, helping reduce the risk of accidents and improve the overall driving experience. ADAS in industrial environments are not very popular even though they provide important benefits such as increased safety, productivity, and cost savings. ADAS can help reduce the risk of accidents and injuries by providing drivers with real-time information about their surroundings, such as obstacles or other hazards. Additionally, ADAS can help reduce the costs associated with accidents, equipment downtime, and other inefficiencies. When ADAS are integrated into off-road vehicles—like forklifts—which are not used in the same ways as other industrial vehicles, like trucks, their use has to be reevaluated. They must be especially made for the job that is done in industrial settings, where there may be dust, noise, workers moving around, etc. Each off-road vehicle can have a specific set of movement patterns. For instance, a forklift is expected to move freely throughout a plant or warehouse, whilst an excavator will mostly be in one place and move in a specific work area. This requires the customization of the ADAS system as well as the feedback mechanisms designed for it. More concretely, forklift driving is a specialized skill that necessitates certain traits for safe and efficient operation. Forklift drivers must be focused, alert, and aware of their surroundings, including people and obstacles. 123 Good hand-eye coordination is essential for maneuvering the forklift accurately, particularly when dealing with heavy loads. The job can be physically demanding, requiring stamina for long hours and repetitive tasks. Safety is of utmost importance, with drivers adhering to guidelines, wearing appropriate protective equipment, and minimizing risks. Patience and attention to detail are necessary, especially when navigating tight spaces or handling delicate loads. One of the most important parts of a system such as the one proposed here, is the interact (...truncated)