A power-aware vision-based virtual sensor for real-time edge computing

Journal of Real-Time Image Processing (2024) 21:103 https://doi.org/10.1007/s11554-024-01482-0 RESEARCH A power‑aware vision‑based virtual sensor for real‑time edge computing Chiara Contoli1 · Lorenzo Calisti1 · Giacomo Di Fabrizio1 · Nicholas Kania1 · Alessandro Bogliolo1 · Emanuele Lattanzi1 Received: 31 January 2024 / Accepted: 20 May 2024 © The Author(s) 2024 Abstract Graphics processing units and tensor processing units coupled with tiny machine learning models deployed on edge devices are revolutionizing computer vision and real-time tracking systems. However, edge devices pose tight resource and power constraints. This paper proposes a real-time vision-based virtual sensors paradigm to provide power-aware multi-object tracking at the edge while preserving tracking accuracy and enhancing privacy. We thoroughly describe our proposed system architecture, focusing on the Dynamic Inference Power Manager (DIPM). Our proposed DIPM is based on an adaptive frame rate to provide energy savings. We implement and deploy the virtual sensor and the DIPM on the NVIDIA Jetson Nano edge platform to prove the effectiveness and efficiency of the proposed solution. The results of extensive experiments demonstrate that the proposed virtual sensor can achieve a reduction in energy consumption of about 36% in videos with relatively low dynamicity and about 21% in more dynamic video content while simultaneously maintaining tracking accuracy within a range of less than 1.2%. Keywords Edge computing · Virtual sensor · Energy-aware object tracking 1 Introduction The edge computing paradigm is gaining momentum thanks to the advent of real-time, low-power Graphical Processing Units (GPUs) and Tensor Processing Units (TPUs) able to run tiny Machine Learning (ML) models on resourceconstrained devices. The shift of paradigm of moving intelligence from the cloud towards the edge provides benefits * Chiara Contoli Lorenzo Calisti Giacomo Di Fabrizio Nicholas Kania Alessandro Bogliolo Emanuele Lattanzi 1 Department of Pure and Applied Sciences, University of Urbino, Piazza della Repubblica 13, Urbino, Italy in terms of privacy, latency, and bandwidth [2, 49]. Of particular interest are applications enabled by combining edge computing with object detection and tracking tasks, such as remote sensing image [11, 24], video surveillance [1, 18], human–computer interaction [48, 54], and autonomous driving [7, 36], to cite a few. Object detection and tracking are two distinct computer vision tasks. Initially, those tasks were mainly carried out on powerful cloud server [12, 37]; subsequently, the idea was to adopt a collaborative approach between the cloud and the edge, where the two tasks are split across the edge computing architecture [6, 33]. A more recent approach envisages the design of efficient machine and deep learning solutions that are lightweight enough to be deployed on resource-constrained edge devices [13, 45]. Three well-known and widely adopted multi-object tracking-by-detection algorithms are Simple Online Realtime Tracker (SORT) [3], DeepSORT [43], and Intersection over Union (IoU) [35]. Many works in the existing literature compare the performance of SORT and DeepSORT algorithms against other state-of-the-art trackers [20, 31, 41, 44, 51]. The same applies to the IoU algorithm [10, 34, 46, 50]. Despite the rich literature, those works lack the Vol.:(0123456789) 103 Page 2 of 12 investigation of the energy efficiency of the tracker algorithms. Only a few papers, e.g., [30], consider skipping some frames for performance improvement. However, the authors consider a fixed number of frames to be skipped and do not explore the energy efficiency of their algorithm. If detection and tracking algorithms are required to run on low-power edge devices, their power consumption requires attention because of the challenges posed by resource constraints [55, 56]. Despite the high attention paid to energy expenditure, most of the literature only focuses on the impact of the detection phase. When the tracking phase is considered, many works, e.g., [47], only consider the single object tracking case. Our work advances existing literature by proposing an adaptive frame rate strategy for the IoU multi-object tracker to provide a real-time, power-aware, energy-efficient tracking algorithm. 1.1 Our contributions In this paper, we propose the real-time vision-based virtual sensors paradigm for energy-efficient multi-object tracking on edge devices. We first thoroughly describe our proposed system architecture, with a particular focus on the Dynamic Inference Power Manager (DIPM). We implement and deploy the virtual sensor and the DIPM to perform extensive experimental measurements to prove the effectiveness and efficiency of our proposed methodology. Specifically, we consider the Single Shot Detector (SSD) MobileNet [9], and the Train Adapt Optimize (TAO) TrafficCamNet [28] as object detectors, and the lightweight Intersection over Union (IoU) tracking algorithm [38]. Our testbed uses the NVIDIA Jetson Nano [27] as an edge device platform, and we tested it on well-known benchmarks based on the MultiObject Tracking (MOT) challenge [25]. Results show that the proposed virtual sensor can achieve a reduction in energy consumption of about 36% in videos with relatively low dynamicity and about 21% in more dynamic video content while simultaneously maintaining tracking accuracy within a range of less than 1.2%. Our contributions are summarized as follows: – Real-time vision-based virtual sensors: a family of synthetic sensors that process data from camera sources and extract anonymous numerical information. The virtual sensor boosts privacy by consolidating data processing on the edge device without sending sensitive data to a centralized server. – Dynamic Inference Power Manager: enhances the virtual sensors by implementing an adaptive frame rate approach to allow energy savings while preserving tracking accuracy. Journal of Real-Time Image Processing (2024) 21:103 – deployment on the NVIDIA Jetson Nano: we highlight the advantages of our methodology compared to conventional non-power-aware edge computing approaches. The rest of the paper is organized as follows: Sect. 2 provides related work on object detection and tracking on edge device platforms. Section 3 describes the design principles of the proposed vision-based virtual sensor and provides a detailed description of the proposed DIPM. In Sect. 4, we provide background on the performance evaluation metrics typically employed to evaluate tracking systems. Section 5 presents the experimental setup and methodology employed to assess the efficacy of our solution. Section 6 discusses the results and findings, while Sect. 7 concludes the paper. 2 Related work This section reviews the relevant literature on object detection and tracking efforts on testbed implementation considering edge hardware platforms. We also review the literature on (...truncated)