A comprehensive survey of image segmentation: clustering methods, performance parameters, and benchmark datasets

Multimedia Tools and Applications https://doi.org/10.1007/s11042-021-10594-9 1174: FUTURISTIC TRENDS AND INNOVATIONS IN MULTIMEDIA SYSTEMS USING BIG DATA, IOT AND CLOUD TECHNOLOGIES (FTIMS) A comprehensive survey of image segmentation: clustering methods, performance parameters, and benchmark datasets Himanshu Mittal1 · Avinash Chandra Pandey1 · Mukesh Saraswat1 · Sumit Kumar2 · Raju Pal1 · Garv Modwel3 Received: 10 June 2020 / Revised: 7 January 2021 / Accepted: 21 January 2021 / © The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature 2021 Abstract Image segmentation is an essential phase of computer vision in which useful information is extracted from an image that can range from finding objects while moving across a room to detect abnormalities in a medical image. As image pixels are generally unlabelled, the commonly used approach for the same is clustering. This paper reviews various existing clustering based image segmentation methods. Two main clustering methods have been surveyed, namely hierarchical and partitional based clustering methods. As partitional clustering is computationally better, further study is done in the perspective of methods belonging to this class. Further, literature bifurcates the partitional based clustering methods into three categories, namely K-means based methods, histogram-based methods, and meta-heuristic based methods. The survey of various performance parameters for the quantitative evaluation of segmentation results is also included. Further, the publicly available benchmark datasets for image-segmentation are briefed. Keywords Image segmentation · Clustering methods · Performance parameters · Benchmark datasets 1 Introduction “A picture is worth a thousand words” is a famous idiom which signifies that processing an image may relieve more information than processing the textual data. In computer vision, image segmentation is the prime research area which corresponds to partitioning of  Raju Pal 1 Jaypee Institute of Information Technology, Noida, Uttar Pradesh, India 2 Amity University, Noida, Uttar Pradesh, India 3 Valeo India Private Limited, Chennai, Tamil Nadu, India Multimedia Tools and Applications an image into its constituent objects or region of interests (ROI). Generally, it assembles the image pixels into similar regions. It is a pre-processing phase of many image-based applications like biometric identification, medical imaging, object detection and classification, and pattern recognition [91]. Some of the prominent applications are as follows. – – – – – Content-based image retrieval: It corresponds to the searching of query-relevant digital images from large databases. The retrieval results are obtained as per the contents of the query image. To extract the contents from an image, image segmentation is performed. Machine vision: It is the image-based technology for robotic inspection and analysis, especially at the industrial level. Here, segmentation extracts the information from the captured image related to a machine or processed material. Medical imaging: Today, image segmentation helps medical science in a number of ways from medical diagnosis to medical procedures. Some of the examples include segmentation of tumors for locating them, segmenting tissue to measure the corresponding volumes, and segmentation of cells for performing various digital pathological tasks like cell count, nuclei classification and many others. Object recognition and detection: Object recognition and detection is an important application of computer vision. Here, an object may be referred to as a pedestrian or a face or some aerial objects like roads, forests, crops, etc. This application is indispensable to image segmentation as the extraction of the indented object from the image is priorly required. Video surveillance: In this, the video camera captures the movements of the region of interests and analysis them to perform an indented task such as identification of the action being performed in the captured video or controlling the traffic movement, counting the number of objects and many more. To perform the analysis, segmentation of the region of interest is foremost required. Though segmenting an image into the constituent ROI may end up as a trivial task for humans, it is relatively complex from the perspective of the computer vision. There are Fig. 1 Challenges in image segmentation a Illumination variation [2] b Intra-class variation [2] c Background complexity [3] Multimedia Tools and Applications number of challenges which may affects the performance of an image segmentation method. Figure 1 depicts three major challenges of image segmentation which are discussed below. – – – Illumination variation: It is a fundamental problem in image segmentation and has severe effects on pixels. This variation occurs due to the different lighting conditions during the image capturing. Figure 1a shows an image that is captured in different illumination conditions. It can be observed that the corresponding pixels in each image contain varying intensity values which pose difficulties in image segmentation. Intra-class variation: One of the major issues in this field is the existence of the region of interest in a number of different forms or appearances. Figure 1b depicts an example of chairs that are shown in different shapes, each having a different appearance. Such intra-class variation often makes the segmentation procedure difficult. Thus, a segmentation method should be invariant to such kind of variations. Background complexity: Image with a complex background is a major challenge. Segmenting an image as the region of interests may mingle with the complex environment and constraints. Figure 1c illustrates an example of such image which corresponds to H&E stained breast cancer histology image. The dark blue color regions in the image represent the nuclei region which is generally defined as the region of interests in histopathological applications like nuclei count or cancer detection. It can be observed that the background is too complex due to which the nuclei regions do not have clearly defined boundaries. Therefore, such background complexities degrade the performance of segmentation methods. Further, the essence of an image segmentation is to represent an image with a few significant segments instead of thousands of pixels. Moreover, image segmentation may be viewed as a clustering approach in which the pixels, that are satisfying a criterion, are grouped into a cluster while dissatisfying pixels are placed in different groups. To exemplify this, consider the images in Fig. 2. The first image consists of some animals on the field. To extract the animals from the background, the ideal result would be to group all the pixels belonging to the animals into the same cluster while background pixels into another cluster as presented in Fig. 2b. However, the pixe (...truncated)