An evaluation of moving shadow detection techniques

An evaluation of moving shadow detection techniques Mosin Russell Ju Jia Zou Gu Fang Shadows of moving objects may cause serious problems in many computer vision applications, including object tracking and object recognition. In common object detection systems, due to having similar characteristics, shadows can be easily misclassified as either part of moving objects or independent moving objects. To deal with the problem of misclassifying shadows as foreground, various methods have been introduced. This paper addresses the main problematic situations associated with shadows and provides a comprehensive performance comparison on up-todate methods that have been proposed to tackle these problems. The evaluation is carried out using benchmark datasets that have been selected and modified to suit the purpose. This survey suggests the ways of selecting shadow detection methods under different scenarios. - Shadows play an important role in our understanding of the world and provide rich visual information about the properties of objects, scenes, and lights. The human vision system is capable of recognizing and extracting shadows from complex scenes and uses shadow information to automatically perform various tasks, such as perception of the position, size, and shape of the objects, understanding the structure of the 3D scene geometry and location and 1 School of Computing, Engineering and Mathematics, Western Sydney University, Locked Bag 1797, Penrith, NSW 2751, Australia. E-mail: M. Russell, ( ); J. J. Zou, j.zou@ westernsydney.edu.au; G. Fang, g.fang@westernsydney. edu.au. Manuscript received: 2016-04-06; accepted: 2016-07-20 intensity of the light sources. For the past decades, researchers working in computer vision and other related fields have been trying to find a mechanism for machines to mimic the human vision system in handling the visual data and performing the associate tasks. However, the problem is far from being solved and all the tasks remain as challenging. Shadows are involved in many low-level computer vision applications and image processing tasks such as shadow detection, removing, extraction, correction, and mapping. In many video applications, shadows need to be detected and removed for the purpose of object tracking [ 1 ], classification [ 2, 3 ], size and position estimation [4], behaviour recognition [ 5 ], and structural health monitoring [ 6 ]. In still image processing, shadow feature extraction is applied to get features that will be useful in object shape estimation [ 7 ], 3D object extraction [ 8 ], building detection [ 9 ], illumination estimation [ 10 ] and direction [ 11 ], and camera parameter estimation [ 12 ]. Shadow detection and correction (i.e., shadow compensation or de-shadowing) involve complex image processing techniques to produce a shadow-free image which can be useful in many applications including reconstruction of surfaces [ 13 ], illumination correction [ 14 ], and face detection and recognition [ 15 ]. In contrary to shadow detection, some applications, such as rendering soft shadow for 3D objects [ 16 ] and creating shadow mattes in cel animation [ 17 ], require rendering shadows to add more spatial details within and among objects and to produce images with a natural-realistic look. Shadow detection and mapping also need to be considered in some recent image processing applications such as PatchNet [ 18 ], timeline editing [ 19 ], and many other visual media processing applications [ 20 ]. Examples of these applications are shown in Fig. 1. Compared with detecting moving shadows in a video, detecting still shadows in a single image is more difficult due to having less information available in a single image than a video. In both cases, various features (such as intensities, colors, edges, gradients, and textures) are used to identify shadow points. In moving shadow detection, these features from the current frame are compared to those of the corresponding background image to find if the features are similar. In still image shadow detection, these features are often used along with other geometric features (such as locations of light sources and object shapes) to detect shadows. In this paper, we focus on shadow detection and removal in image sequences (also called moving shadow detection) and introduce a novel taxonomy to categorize the up-to-date existing moving shadow detection methods based on various metrics including moving objects and their cast shadow properties, image features, and the spatial level used for analysing and classification. Listed below are the major contributions of this paper: • The main problematic situations associated with shadows are addressed and common datasets are organized and classified based on these problematic situations. • A unique way to analyse and classify most key papers published in the literature is introduced. • A quantitative performance comparison among classes of me (...truncated)