GPU-based key-frame selection of pulmonary ultrasound images to detect COVID-19

Journal of Real-Time Image Processing (2024) 21:113 https://doi.org/10.1007/s11554-024-01493-x RESEARCH GPU‑based key‑frame selection of pulmonary ultrasound images to detect COVID‑19 Emanuele Torti1 · Marco Gazzoni1 · Elisa Marenzi1 · Francesco Leporati1 Received: 18 April 2024 / Accepted: 5 June 2024 © The Author(s) 2024 Abstract In the last decades, technological advances have led to a considerable increase in computing power constraints to simulate complex phenomena in various application fields, among which are climate, physics, genomics and medical diagnosis. Often, accurate results in real time, or quasi real time, are needed, especially if related to a process requiring rapid interventions. To deal with such demands, more sophisticated approaches have been designed, including GPUs, multicore processors and hardware accelerators. Supercomputers manage high amounts of data at a very high speed; however, despite their considerable performance, their limitations are due to maintenance costs, rapid obsolescence and notable energy consumption. New processing architectures and GPUs in the medical field can provide diagnostic and therapeutic support whenever the patient is subject to risk. In this context, image processing as an aid to diagnosis, in particular pulmonary ultrasound to detect COVID-19, represents a promising diagnostic tool with the ability to discriminate between different degrees of disease. This technique has several advantages, such as no radiation exposure, low costs, the availability of follow-up tests and the ease of use even with limited resources. This work aims to identify the best approach to optimize and parallelize the selection of the most significant frames of a video which is given as the input to the classification network that will differentiate between healthy and COVID patients. Three approaches have been evaluated: histogram, entropy and ResNet-50, followed by a K-means clustering. Results highlight the third approach as the most accurate, simultaneously showing GPUs significantly lowering all processing times. Keywords ResNet · K-means · High-performance computing · Key-frame selection · Artificial intelligence · Machine learning 1 Introduction The SARS-CoV-2 virus belongs to the coronavirus family and is responsible for the COVID-19 infection, which involves the respiratory apparatus and is transmitted through drops produced when coughing, sneezing or simply talking * Elisa Marenzi Emanuele Torti Marco Gazzoni Francesco Leporati 1 Department of Computer, Electrical and Biomedical Engineering, University of Pavia, Via Ferrata 5, 27100 Pavia, Italy and breathing. The incubation period lasts usually 4–5 days [1]. If symptoms develop, after about a week the illness can either heal or worsen into a serious condition, developing dyspnoea due to hypoxaemia, quickly followed by respiratory insufficiency that evolves into an acute respiratory distress syndrome. This can be observed as a pulmonary lesion characterized by inflammation and loss of pulmonary tissue [1]. In certain cases, pneumonia is present, producing a decrease in oxygen saturation and alterations, such as anomalies of frosted glass, stain stabilization and interlobular involvement made visible thanks to radiations and other imaging techniques [2]. This illness has induced scientists to develop rapidly reliable diagnostic approaches to reduce both the spreading of the infection and serious complications. The traditional diagnostic methodology to detect and monitor pneumonia is computed tomography (CT) of the thorax; COVID-19 Vol.:(0123456789) 113 Page 2 of 14 patients in particular show specific features: frosted or thin reticular opacities, reticulation, vascular thickening, traction bronchiectasis, bilateral involvement, predominant inferior lung involvement and multifocal distribution [3]. CT scans show high sensitivity and specificity in detecting COVID19 pneumonia; however in multiple studies, both symptomatic and asymptomatic patients did not show identifiable anomalies. Besides, in a considerable number of children undergoing CT, no pathological signs could be detected [4]. Therefore, new diagnostic tools are required, since many patients with COVID-19 pneumonia do not show alterations, especially in the first stages of the disease [5]. Pulmonary ultrasound is a non-invasive technique to diagnose and follow up pulmonary interstitial syndrome, since it is able to identify and discriminate between healthy portions of pleura from irregularities, nodules and thickenings [6]. This kind of approach has shown high levels of precision in the diagnosis of pneumonia, even compared to traditional and consolidated methods like CT scans. Therefore, nowadays it is considered an alternative to CT for the diagnosis of interstitial diseases and pneumonia, as well as acute respiratory distress syndrome, with accuracies always higher than 90% and higher values of sensitivity and specificity also [7–9]. Hence, ultrasound imaging represents a relevant technique both in the diagnostic and therapeutic fields, thanks to the opportunity to detect in real time the dynamics of organs’ movements and details of blood flow, as well as their low costs without the use of radiation. In this work, a database of pulmonary ultrasound videos has been made available by the IRCCS San Matteo General Hospital of Pavia, Italy, where recordings can vary greatly in terms of the number of frames, duration (due to the specific examination and the subject under test) and resolution (because of the type of instrument in use). Nonetheless, such data represents a fundamental resource for a better understanding of pulmonary issues and improvement of diagnostic approaches. The aim of this work is to provide an innovative way to diagnose the presence of COVID-19 and determine its severity through a non-invasive method, based on pulmonary ultrasound. Keyframe selection is used to select the most informative frames of acquired videos, followed by a ResNet-50 and K-means to highlight diagnostic patterns and then group such frames in terms of their similarity. This methodology has been compared with consolidated approaches to determine the most appropriate frames and severity parameters. The results confirm the ability to correctly detect diagnostic evidence and severity of COVID-19. 1.1 Related works Imaging modalities such as chest X-ray, computed tomography scans and ultrasound are used for rapid and Journal of Real-Time Image Processing (2024) 21:113 precise COVID-19 diagnoses; however, processing such images is time-consuming and susceptible to human error. Therefore, artificial intelligence (AI) methods and in particular deep learning (DL) models provide highperformance results, since they automate all stages of feature extraction, selection and classification [10]. More specifically, numerous studies have demonstrated that both CT and lung ultrasound represent the most appropriate diagnostic to (...truncated)