Transfer learning based cascaded deep learning network and mask recognition for COVID-19

World Wide Web https://doi.org/10.1007/s11280-023-01149-z Transfer learning based cascaded deep learning network and mask recognition for COVID-19 Fengyin Li1 · Xiaojiao Wang1 · Yuhong Sun1 · Tao Li1 · Junrong Ge1 Received: 4 August 2022 / Revised: 8 November 2022 / Accepted: 1 February 2023 © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023 Abstract The COVID-19 is still spreading today, and it has caused great harm to human beings. The system at the entrance of public places such as shopping malls and stations should check whether pedestrians are wearing masks. However, pedestrians often pass the system inspection by wearing cotton masks, scarves, etc. Therefore, the detection system not only needs to check whether pedestrians are wearing masks, but also needs to detect the type of masks. Based on the lightweight network architecture MobilenetV3, this paper proposes a cascaded deep learning network based on transfer learning, and then designs a mask recognition system based on the cascaded deep learning network. By modifying the activation function of the MobilenetV3 output layer and the structure of the model, two MobilenetV3 networks suitable for cascading are obtained. By introducing transfer learning into the training process of two modified MobilenetV3 networks and a multi-task convolutional neural network, the ImagNet underlying parameters of the network models are obtained in advance, which reduces the computational load of the models. The cascaded deep learning network consists of a multi-task convolutional neural network cascaded with these two modified MobilenetV3 networks. A multi-task convolutional neural network is used to detect faces in images, and two modified MobilenetV3 networks are used as the backbone network to extract the features of masks. After comparing with the classification results of the modified MobilenetV3 neural This article belongs to the Topical Collection: Special Issue on Privacy and Security in Machine Learning Guest Editors: Jin Li, Francesco Palmieri and Changyu Dong. B Tao Li Fengyin Li Xiaojiao Wang Yuhong Sun Junrong Ge 1 School of Computer Science, Qufu Normal University, Rizhao City, Shandong Province, China 123 World Wide Web network before cascading, the classification accuracy of the cascading learning network is improved by 7%, and the excellent performance of the cascading network can be seen. Keywords COVID-19 · Mask recognition · Cascade network · MobilenetV3 · Transfer learning 1 Introduction Since the outbreak, COVID-19 has spread to all continents around the world, becoming a global pandemic. While causing serious harm to people’s lives and health, COVID-19 has also had a major impact on the economy, society and politics [1]. The spread of COVID-19 is very broad. The virus is spread by aerosols or droplets formed when infected people talk, cough, or sneeze. When healthy people are in close contact with infected people, healthy people may be infected with the virus through direct contact with aerosols or droplets. Therefore, wearing a mask is a powerful and effective way to avoid infection and spread of COVID-19 [2]. The World Health Organization (WHO) calls on people to wear masks in workplaces, schools and shops without adequate ventilation, and areas where COVID-19 spreads should strictly abide by this guideline [3]. At present, the entrances of public places are equipped with cameras to detect whether pedestrians are wearing masks. However, the current detection system is not robust, as long as pedestrians cover their faces, they can pass the system detection, and pedestrians can pass through the gate. Many pedestrians are not aware of safety and do not wear masks when they go out. When entering large places, they enter relevant places through system detection by covering their faces with their hands or covering their faces with scarves. There are also frugal people who pass the detection system by wearing cotton masks that are less protective but can be used multiple times [4]. Such actions to deceive the detection system are called presentation attacks, and the biometrics or related instruments used in presentation attacks are called Presentation Attack Instruments (PAI). The ability of a detection system to identify PAI is called Presentation Attack Detection (PAD) [5]. Generally, the stronger the PAD of a system, the more types of PAIs the system can recognize. An ideal PAD system should be able to detect all known PAIs, as well as new unknown PAIs that appear in the future [6]. An ideal mask recognition system should be able to detect all obstructions. Therefore, at the entrance of public places, the detection system should be able to distinguish between pedestrians wearing masks and other coverings. In this paper, a cascaded [7] network model for identifying mask types is proposed. This model can process the images detected by the camera to identify whether the face in the image wears a mask and the type of mask worn. Since the main purpose of the detection system is to allow pedestrians wearing masks to conduct relevant places, the main purpose of the cascade learning network constructed here is to improve the ability of the system to identify masks, not to improve the ability of the system to identify specific species of PAIs [8]. In this paper, a cascaded three-stage mask classification network is constructed by sequentially inputting images into a Multi-task convolutional neural network (MTCNN) [9] and two modified MobilenetV3 networks [10–12]. This cascaded network is used to classify images of wearing masks, wearing other coverings, and not wearing masks. The main contributions of the paper are listed below. 123 World Wide Web • By modifying the lightweight network architecture MobilenetV3, updating the activation function of the MobilenetV3 output layer and the structure of the model, two modified MobilenetV3 networks are obtained. • Aiming at the problems of many parameters, long training time and large amount of computation in deep neural networks, transfer learning is applied to the training process of MTCNN and two modified MobilenetV3 networks respectively, and a deep neural network based on transfer learning is proposed. When the network model is trained, the underlying parameters use the transferred parameters, which reduces the training time of the model and the calculation amount of the model. • By cascading MTCNN with two modified MobilenetV3 networks, a cascaded deep learning network based on transfer learning is proposed. First use MTCNN to detect faces, excluding the influence of the background and clothing of pedestrians in the images on the subsequent classification of masks. Then use the first modified MobilenetV3 network for mask detection, and finally use the second modified MobilenetV3 network for mask recognition. After double screening by two modified MobilenetV3 networks, the final accuracy of t (...truncated)