Ellipticity and Circularity Measuring via Kullback–Leibler Divergence

Journal of Mathematical Imaging and Vision, Dec 2015

Using the Kullback–Leibler divergence we provide a simple statistical measure which uses only the covariance matrix of a given set to verify whether the set is an ellipsoid. Similar measure is provided for verification of circles and balls. The new measure is easily computable, intuitive, and can be applied to higher dimensional data. Experiments have been performed to illustrate that the new measure behaves in natural way.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:


Ellipticity and Circularity Measuring via Kullback–Leibler Divergence

J Math Imaging Vis Ellipticity and Circularity Measuring via Kullback-Leibler Divergence Krzysztof Misztal 0 Jacek Tabor 0 0 Faculty of Mathematics and Computer Science, Jagiellonian University , Ł ojasiewicza 6, 30-348 Kraków , Poland Using the Kullback-Leibler divergence we provide a simple statistical measure which uses only the covariance matrix of a given set to verify whether the set is an ellipsoid. Similar measure is provided for verification of circles and balls. The new measure is easily computable, intuitive, and can be applied to higher dimensional data. Experiments have been performed to illustrate that the new measure behaves in natural way. Kullback-Leibler divergence; Circularity measure; Ellipticity measure; Image processing 1 Introduction Human image analysis still poses huge challenges for a scientist. Automatic computer object recognition and interpretation of an image is crucial in building excellent image analysis software, especially for extracting higher-level information. Real-life applications forced to develop the idea to describe the object characteristics by using a set of numbers, thus enabling a spectrum of numerical quantifications. Many shape descriptors were created and used [ 1,12 ]. Some of them have generic purpose, such as the Fourier descriptors [3] or moment invariants [ 10,15 ]. On the other hand, for the specific purpose of classification, several shape descriptors are useful for describing and differentiating a variety of objects: convexity [23], rectangularity [ 24 ], linearity [ 27 ], symmetry [ 29 ], etc. Note that, due to the diversity of shapes, descriptors have applications in various areas such as computer science, medicine, biology, and robotics. Recently, there have been several applications that concentrate on finding and classifying circular and elliptical objects in images. They range from identifying traffic light and sign [ 4 ], to face detection [ 16 ]. In this paper we define a new measure for two important descriptors: circularity and ellipticity in arbitrary dimensions. Future applications might include, for example, the identification of sick individual cells based on their boundaries in medical imaging—compare Fig. 1. Our motivation, from the theoretical point of view, can be illustrated by the following problem. Problem 1 If the actual area occupied by an object can be estimated using the well-known area formula πr1r2, it has a good chance of being an ellipse, or, if πr 2—a circle. Obviously, the questions how to estimate the major and minor ellipse radii r1, r2 (or r ), or how to formalize “good chance” still need to be answered. To develop the solution which will solve the presented problem, we consider closely the existing ones, namely the methods which verify whether the set is an ellipsoid, and which differentiate the shapes between ellipses and circles. The work presented in this article aims to partially generalize the methods presented by Žunic´ et al. in work [ 31 ], Žunic´ and Žunic´ in [ 30 ], and Rosin in [ 24 ] which describe circularity and ellipticity measures. The reason to choose these methods is their performance superiority in the case of shape boundary defects compared to the other standard methods, namely, the behavior of these measures (i.e., numerical shape characteristics) can be relatively easily understood and their behavior can be reasonably predicted. The aforementioned articles describe explicit formula which use the first two Hu moments invariants to evaluate how much a planar shape differs from a circle or an ellipse. Detailed information of those measures—of circularity CH and ellipticity EH and EI —are presented in Sect. 2 of this article. Our aim is to deal with the problem of ellipsoids and balls recognition in the general case of RN . The main idea of our approach uses the information theory concept called Kullback–Leibler divergence and can be reduced to verifying whether the value of is maximal for S ⊂ RN , where S denotes the covariance matrix1 of S and λN the N -dimensional volume2 (Lebesgue measure) of S. The main result of this work presented in Theorem 2 gives an estimation of (1) in the case of circles and ellipses. This allows to derive the condition to test if a given set is ellipticallike EN or circular-like CN . Hence, they can be used as a (1) 1 If set S is discrete, namely S = {xi }iN=1, then the covariance matrix equals S = N1 iN=1(xi − μ)(xi − μ)T , where μ = N1 iN=1 xi is a mean of set S. In the general case by the covariance matrix of a set, we understand the covariance of the uniform normalized density on S. 2 If a set S is a discrete subset of ZN , then in practical considerations we view S a discrete representation of the set S1 = S + [−1/2, 1/2]N . Thus, the value λ(S1) equals card(S) and the covariance matrix S1 equals S + 112 I . measure of ellipticity and circularity. Our measures were tested on several examples from [ 30, 31 ]. Experiments verify many advantages of our approach, e.g., behavior consistent with the human intuition and its invariance in similarity transformation. Moreover, our measures can be applied to higher dimensional data (see Figs. 9, 10, 11). This paper is organized as follows. In the next section the foundations for the state of art are introduced. In Sect. 3 we briefly describe the main result of this work with the sketch of the proof. In Sect. 4 we set up notation and terminology for the Kullback–Leibler divergence and cross-entropy. In Sect. 5 we provide the formula for circularity and ellipticity measurement. Comments and conclusions can be found in the last section. 2 State of the Art In this section several most standard measures of circularity and ellipticity are mentioned. Those methods range over (0, 1] and give the measurement equal to 1 if and only if the measured shape is a circle or an ellipse. Let us consider an arbitrary set S ⊂ R2. 2.1 Circularity Measure Form the historical point of view, the first circularity measure was introduced in [ 6 ] an later in the digital plane in [ 13 ]. It is given by 4π · λ2(S) Cst (S) = (perimeter_of_S)2 , where λ2(S) is the area of the set S (compare with [ 26 ]). Geometrical moments can also be used in the circularity measure. Example of such approach can be found in [ 31 ]. Circularity can be measured by the quantity (μ0,0(S))2 CH (S) = 2π(μ2,0(S) + μ0,2(S)) where the centralized ( p, q)-moment μ p,q (S) of a planar set S is μ p,q (S) = S p x − μx (S) y − μy (S) q dx d y for (μx (S), μy (S)) – the centroid of S [ 15 ]. Other examples of methods for measuring the circularity can be found in [7, 13, 14, 22]. 2.2 Ellipticity Measure The first approach is based on moment invariants. Since any ellipse can be obtained by applying an affine transform to a circle, we use the simplest affine moment invariant [ 9 ] (based on the central moments μ p,q ) of the circle to characterize ellipses I1(S) = μ2,0(S)μ0,2(S) − μ12,1(S) , where the centralized ( p, q)-moment of a planar set S is given by μ p,q (S). To discriminate the shape and measure of ellipticity, we use EI (S) = 16π 2 I1(S), if I1(S) ≤ 161π2 , 1 16π2 I1(S) , otherwise, (compare with [ 24 ]). The second approach is based on the first two Hu invariant moments [10,15]. The ellipticity measure of a given shape S can be computed by the formula [30]: EH (S) = 2π 2 I1(S) · 4I2(S) + 1/π 2 − 2I2(S) where – I1(S) = m2,0(S) + m0,2(S), – I2(S) = (m2,0(S) − m0,2(S))2 + 4(m1,1(S))2, for the geometric moments of a given shape defined by m p,q (S) = S x p yq dx dy. Other examples of methods for measuring the ellipticity can be found in [8,21,22,25,26]. −1 , 3 Main Theorem In this section we present the main result of this paper that a set S is an ellipsoid if the uniform probability density on it has minimal Kullback–Leibler divergence—a fundamental equation of information theory that quantifies the proximity of two probability distributions (a brief summary and the proof is presented in further part of this work). Using Kullback–Leibler divergence we show that it is enough to know three moments of the object (in R2) to check if the given set is an ellipse. This condition for an arbitrary set S ⊂ RN reduces to verifying whether the value of λN (S)/ det( S) ((N + 2)π )N/2 is maximal, where S denotes the covariance of the uniform probability measure on S, and equals3 For N = 1 (line) and N√ = 2 (the situation on the plane), (esNitusaimtiopnli fiinesthtroeee1di=me2nsi3onaanldspea2c=e) w4πe ,gwetheil3e=fo2r0N√35=π .3 We obtain also analogous estimations for circles and balls in RN . Given a symmetric positive matrix , we recall that the Mahalanobis distance [ 18 ] is given by x − y := (x − y)T −1(x − y). Thus, our main result (compare with Corollaries 1 and 2) may be stated as follows: Theorem 1 Let S ⊂ RN with mean mS and covariance be given. S – Then Γ (N /2 + 1) λN (S) EN (S) := ((N + 2)π )N/2 · √det( S) ≤ 1, where the equality holds if S is an ellipse. If this is the case, then S = B S (mS, √N + 2). – Then Γ (N /2 + 1) λN (S) CN (S) := ((N + 2)π/N )N/2 · (tr( S))N/2 ≤ 1, where the equality holds if S is a circle. If this is the case then S = B(mS, NN+2 tr( S)). We postpone the proof till Sect. 5. However, the basic idea consists of the following steps: – we first observe that we can restrict to the case when the mean of S is centered at zero and the covariance equals to identity; – next we fit to the data optimal uniform density on a ball B(0, R) with R such that the volume of S equals to volume of B(0, R); 3 By Γ (x), for x > 0, we denote the Gamma function which is an extension of the factorial function. Fig. 2 Ellipse vs. circle—comparison of the measure values to test if the object is elliptical-like (E2) or circle-like (C2) (higher value means that the indicated shape kind describes the object better) – last we show that if S would contain elements outside of B(0, R), then by “moving” those elements inside of B(0, R), we would increase the value of the respective Kullback–Leibler divergence. For the convenience of the reader, we now discuss the situation on the plane. We consider S ⊂ R2 with mean mS and covariance S. Then (2) (3) λ2(S) E2(S) := 4π √det( S) λ2(S) C2(S) := 2π tr( S) . , Figure 2 presents the example values of E2 and C2 for given sets. Under the above definition, parameter E2 is invariant to affine transformations, while C2 is invariant to isometric transformations (compare with Theorem 3). Remark 1 If S is an ellipse, then one can easily verify that 4π √det( S) equals its area. Thus, we see that (2) is a realization of the idea given in Problem 1. Analogously, if S is a circle, then its area equals 2π tr( S), and consequently (3) gives a formalization of an analogue of Problem 1 for circles. Directly from Theorem 1 (namely, Eqs. (2) and (3)), we can compare the new measures with the measures recalled in state of the art of this article (Sect. 2). Observation 1 For an arbitrary set S ⊂ R2: – C2(S) = CH (S); – E2(S) ≤ a ⇒ EI (S) ≤ a2 for a ∈ (0, 1]. Proof Recall now that the covariance matrix of the S is given by [ 10 ] 1 S = μ0,0(S) μ2,0(S) μ1,1(S) μ1,1(S) μ0,2(S) Thus, by definition of CH (S) λ2(S) C2(S) = 2π tr( S) μ0,0(S) = 2π μ0,0(S) (μ2,0(S) + μ0,2(S)) 1 μ02,0(S) = 2π(μ2,0(S) + μ0,2(S)) = CH (S) which completes the proof of the first equality. For the second property, we have E2(S) = μ0,0 4π (μ2,0(S)μ0,2(S) − μ12,1(S))/μ20,0(S) = = 16π 2 I1(S) . 1 16π 2 · μ2,0(S)μ0,2(S) − μ12,1(S) 1/2 μ04,0(S) −1/2 Consequently, by Theorem 2 we obtain 16π 2 I1(S) 1 1 which implies that I1(S) ≤ 16π2 for an arbitrary set S ⊂ R2. Hence, E2(S) = (EI (S))1/2 which completes the proof. 1/2 ≤ Consequently, the authors’ approach for the two-dimensi onal data leads to the same conclusions as indexes EI [ 24 ] and CH [ 31 ]. At the end of this section we discuss how the above mea sures can be adapted for discrete (finite) sets. In general in our opinion, it is a nontrivial problem; luckily we can easily deal with the case when S is a discrete subset of δ · ZN (δ > 0). Then instead of S we consider the set 1 1 N S˜ := S + δ · [ − 2 , 2 ] . Then, as one can easily check λN (S˜) = δ N · card(S) and S˜ = S + δ N +2 12 I, (4) where by S on the RHS we understand the standard covariance of the discrete set. Examples of discretization for a few δ values are presented in the Fig. 3. The discrete case appears to be important also in our numerical experiments, where we approximate the “exact” shape S ⊂ RN by its discrete approximation given by Sδ := {x ∈ δZN : x ∈ S} with δ → 0. Observe that for compact sets with nonzero Lebesgue measure all the moments of Sδ converge, with δ → 0, to the respective moments of S. Consequently, the same holds for constants EN and CN . Surprisingly this is not the case for Cst , where even for the case of unit circle Cst (B(0, 1)δ) → Cst (B(0, 1)) as δ → 0, which follows from the fact that the length of the boundary of the discrete approximation of the set4 usually does not 4 We measure the boundary length as a sum of distance of consecutive middle points of the squares which form the boundary of the set, see Fig. 3. converge to the exact length of the boundary—it occurs that in the optimal value is obtained for octagon instead of a circle. This follows from the fact that when we measure the length of the discretization Sδ of the set S, by following the discrete boundary we can move only in the directions which have the angle which is the multiplicity of π/4. Thus although Cst (B(0, 1)) = 1, 1 Cst (B(0, 1)δ) ≤ Cst (octagon) = 8 1 + √2 π ≈ 0.948059. Remark 2 The Cst uses one of the most popular and standard approach to circularity measure which is derived from the relation between the shape of the area and the length of its perimeter. As one can show, which can be observed also in above examples, this measure stabilizes on the octagon where it achieves the highest value. This is caused by that fact that in the calculation of the boundary of given discrete shape we can “move” only according to lines which form the angle which is a multiplicity of π/4 with the axis, see Fig. 3c for illustration. Since this measure was for a long time successfully applied for circle discovery on images, we conclude that from the practical point of view, octagon presents a sufficient numerical approximation of the circle in most commonly encountered application. 4 Kullback–Leibler Divergence and Cross-Entropy 4.1 Basic Definitions on Kullback–Leibler Divergence We now remind the reader of the concept of differential entropy which is the entropy of a continuous random variable [ 5 ]. Definition 1 The differential entropy h( f ) of a continuous random variable with a density function f : RN → R+ is defined as h( f ) = − Differential entropy is also related to the shortest description length, and is similar in many ways to the well-known entropy of a discrete random variable. Since it extends the idea of Shannon entropy, a measure of the expected value of the information in the message, to continuous probability distributions. The value of differential entropy depends only on the probability density of the random variable [ 5 ]. In this paper we shall abbreviate differential entropy as entropy. Lets now calculate differential entropy for simplest density—uniform density. Example 1 (Uniform distribution) Consider the random variable distributed uniformly on the set S ⊂ RN , so that its density uS is 1/λN (S) on S and 0 elsewhere. Then its differential entropy is h(uS) = − 1 S λN (S) ln 1 Since our aim is to study mainly uniform densities, for a measurable S ⊂ RN with finite and nonzero measure to shorten notation, we will use the symbol uS in the place of the uniform probability density 1 uS := λN (S) 1S on S, where 1S(x ) := 1, if x ∈ S, 0, if x ∈/ S. As a consequence, we will write μS and mean and covariance of uS. S to denote the Remark 3 In general case one could consider various densities, not just the uniform one. However in practical applications, Gaussian densities are typically considered, as they are easy to work with. In many practical cases, the methods developed under the assumption that data have normal distribution work quite well even when the density is not normal. Furthermore, the central limit theorem provides a theoretical basis for why it has such a wide applicability. Therefore, this density approximates many natural phenomena so well, and it has developed into a standard of reference for many probability problems. As an excellent example we can refer the reader to [ 11 ] where Gaussian distributions were used for modeling contours and have been applied for shape retrieval. Since in this paper we focus on circular and elliptical shapes, to detect them we could theoretically use densities which have ellipses or circles as level sets. We have decided to use Gaussian ones, while for them we have accurate, explicit, and numerically efficient formulas for the estimation of their parameters. The differential entropy of Gaussian density is considered in following example. Example 2 (Gaussian distribution [ 28 ]) For the multivariate Gaussian distribution, the entropy goes as the log determinant of the covariance; specifically, the differential entropy of a N -dimensional random variable with the density function Nμ, (x ) := is given by the formula 1 (2π )N det( ) exp − 21 x − μ 2 h(Nμ, ) = − RN Nμ, (x ) ln Nμ, (x )dx N 1 = 2 ln(2π e) + 2 ln(det( )). We can now proceed to Kullback–Leibler divergence which is the “cost” associated with selecting a distribution q from distribution family Q to approximate the true distribution p [ 5 ]. Definition 2 The Kullback–Leibler divergence (or relative entropy) DK L ( p q) between two densities p and q is defined by RN DK L ( p q) := ln( p(x )/q(x )) · p(x ) dx . For a family of densities Q, the Kullback–Leibler divergence is given by DK L ( p Q) := qi∈nQf DK L ( p q). DK L is nonnegative in p and q, zero if the distributions match exactly and can potentially equal infinity. However, since the Kullback–Leibler divergence is a nonsymmetric information theoretical measure of distance of densities p from q, namely DK L ( p q) = DK L (q p), it is not strictly a distance metric. However, there are some natural modifications which deal with this problem, e.g., [ 2,17 ]. By introducing next definition—cross-entropy, we can simplify the DK L ( p q) for arbitrary densities p and q. Definition 3 The cross-entropy H ×( p q) of two continuous probability densities p and q is defined as H ×( p q) = − RN p(x ) ln q(x )dx . It is worth specifying that cross-entropy is a variant of the entropy definition that allows us to compare two probability distributions for the same random variable. We treat the first argument as the “target” probability distribution and the second as the estimated one for which we are trying to evaluate how well it “fits” the target. In the case when p has finite entropy, by Definition 3 we can use the following equation DK L ( p q) = − Thus, DK L ( p q) is the measure of the additional cost to pay for the model mismatch—the difference between the descriptions of the random variables by q and by p. 4.2 Kullback–Leibler Divergence Between Uniform and Gaussian Densities We proceed to comparison of uniform and normal distributions by relative entropy. Let G denote the set of all Gaussian densities. One can easily verify that for arbitrary density f and Gaussian density g ∈ G, we have H ×( f g) = H ×(G[ f ] g), where G[ f ] denotes Gaussian density with the same mean and covariance as f . This means that the Kullback–Leibler divergence DK L ( f G) is realized for g = G[ f ]. We will now show the formula for the Kullback–Leibler divergence of uniform densities. Observation 2 For a given uniform density uS on the set S ∈ RN and the Gaussian densities, we have N 1 DK L (uS G) = 2 ln(2π e) + 2 ln(det( S)) − ln(λN (S)). Proof Clearly DK L (uS G) = H ×(uS G) − h(uS) = H × (G[uS] G)−ln(λN (S)) = H ×(G[uS] G[uS])−ln(λN (S)) = h(G[uS]) − ln(λN (S)) = N2 ln(2π e) + 21 ln(det( S)) − ln(λN (S)). A crucial role in our investigation will be played by the following constant dN := DK L (uBN (0,1) G), where BN (0, 1) denotes the unit ball in RN and G denotes the set of all Gaussian densities. From the scientific point of view, it describes how good the compression of uniform density on the unit ball by Gaussians is compared to the optimal compression. Since dN plays the basic role in our considerations, let us calculate it in the following series of examples. Example 3 Consider a uniform probability density on the unit ball BN (0, 1) in RN . Clearly mBN (0,1) = 0. Its covariance matrix BN (0,1) = [si j ] shall be computed. Obviously, si j = 0 if i = j . Consider the case i = j . Since sii = λN (BN1(0,1)) xi2dx , the constant s = sii is well defined BN (0,1) and does not depend on the choice of i , and 1 N s = s11 + · · · + sN N = λN (BN (0, 1)) BN (0,1) 2 r N −1λN −1(∂BN (0, 1))dr r · 1 1 = λN (BN (0, 1)) 0 1 λN −1(∂BN (0, 1)) . = N + 2 λN (BN (0, 1)) Since 1 λN (BN (0, 1)) = 1 · r N −1λN −1(∂BN (0, 1))dr 0 1 = N λN −1(∂BN (0, 1)), we obtain BN (0,1) = N1+2 I, where I denotes the identity matrix. As a direct consequence, we derive B(0,√N +2) = I. BN (0,1) = N1+2 I, Example 4 From Example 3 we have which implies that N H ×(uBN (0,1) G) = 2 ln 2π e N + 2 . Comparing this with h(uBN (0,1)) = ln(λN (BN (0, 1))) = N2 ln π − ln Γ ( 2 + 1), we obtain N N dN = 2 ln Consequently, 5 Optimal Estimations and Main Results and 5.1 Basis for the Simple Case We shall now show that dN gives a lower bound on the compression of the uniform densities, namely, we will calculate the mismatch of the optimal model given by the uniform density of S ⊂ RN and the approximation given by normal densities G, measured by the Kullback–Leibler divergence (see Definition 3). Proposition 1 Let S ⊂ RN be such that μS = 0 and S = I. Then DK L (uS g) ≥ dN for g ∈ G with the equality holding if S = B(0, √N + 2) and g = N (0, I). Proof By the observation from the previous section, we know that DK L (uS G) is realized for N (0, I): DK L (uS G) = H ×(uS N (0, I)) − h(S). This means that if g ∈ G is arbitrary, then DK L (uS g) ≥ DK L (uS N (0, I)) with the equality holding if g = N (0, I). Consequently, we may reduce to the case g = N (0, I). theCnlbeyartlhye, iEf xSa mispaleb3a,llwceenotbetraeidn aSt =zerBo(s0u,c√h Ntha+t 2)S. = I, Consider now the case when S is not a ball centered at zero (modulo a set of zero measure) and let B(0, r ) denote a ball centered at zero with the same Lebesgue measure as S. We will show that dN ≤ DK L (B(0, r ) g) = H ×(B(0, r ) g) − h(B(0, r )) = H ×(B(0, r ) N (0, I)) − ln(λN (B(0, r ))) < H ×(uS N (0, I)) − ln(λN (S)) = DK L (uS G), which will complete the proof. Since by the assumptions λN (B(0, r )) = λN (S), it is sufficient to show that H ×(uB(0,r) N (0, I)) < H ×(uS N (0, I)). (6) Since H ×(uS N (0, I)) 1 = λN (S) S x 2 2 N − 2 ln(√2π ) dx 1 = λN (BN (0, r )) S x 2 N 2 dx − 2 ln(√2π ), H ×(uB(0,r) N (0, I)) 1 = λN (BN (0, r )) B(0,r) 2 dx − N2 ln(√2π ) to verify (6) it is sufficient to show that the following inequality is true BN (0,r) x 2dx < x 2dx . S Let C = B(0, r )\S, D = S\B(0, r ). Clearly from the assumptions both C and D have nonzero measures. Since λN (C ) + λN (B(0, r ) ∩ S) = λN (B(0, r )) = λN (S) = λN (D) + λ(B(0, r ) ∩ S), hence, measures of C and D are equal. To prove (7) it is sufficient to observe that since C ⊂ B(0, r ) and D ⊂ RN \B(0, r ), we have x 2dx < C r 2dx = D r 2dx < x 2dx , C B(0,r) and therefore x 2dx = < = B(0,r)∩S B(0,r)∩S S x 2dx . D C D x 2dx + x 2dx + x 2dx x 2dx (7) 5.2 Main Results: New Measures We shall now broaden the previous theorem to a more general case. This will provide the grounds for defining the new measures. Theorem 2 Let S ⊂ RN with mean μS and covariance Then by applying transformation x → ( S )−1/2 x , we reduce the theorem to the case when S = I. Proposition 1 completes the proof. It is worth observing that the above allows one to check whether a given set is a ball or a ellipsoid with given radius and covariance. Corollary 1 Let S ⊂ RN with mean μS and covariance Then S . λN (S)/ det( S ) ≤ ((N + 2)π )N /2 , with the equality holding if S is an ellipsoid. In such case S = B S (μS , √ N + 2). Proof From Observation 2 and Example 4 DK L (uS G) = ln (2π e)N /2 · √det λ(S) S N ≥ 2 ln with the equality holding if S is an ellipsoid, after trivial calculations, we obtain that for every S Γ (N /2 + 1)λN (S) ((N + 2)π )N /2 · √det( S ) ≤ 1 with the equality holding if S is an ellipsoid. We denote for an arbitrary S ⊂ RN : Γ (N /2 + 1) λN (S) EN (S) := ((N + 2)π )N /2 · √det( S ) , which is an ellipticity measure. By considering the family of all spherical Gaussians G(·I), that is the Gaussians with covariance proportional to identity, we obtain the formula for N -balls identification. Corollary 2 Let S ⊂ RN with mean μS and covariance Then tr( S ) . Proof By Observation 2 and Example 4 DK L (uS G(·I )) = ln 2π e N N /2 tr( S ) N /2 1 λ(S) N ≥ 2 ln with accuracies described by CN and EN , respectively. This highlights that the given theory can be used as a technique for reducing the complexity of a given object (e.g., in the motion tracking5)—see Fig. 4. Moreover, in the case of an elliptical shape we can obtain the object orientation in space by the S matrix. 6 The new measure properties in simple illustrations The following theorem summaries the desirable properties of EN and CN . Theorem 3 The ellipticity measure EN (S) and circularity measure CN (S) of given nonempty set S ⊂ RN satisfies 5 According to the values of CN and EN we can decide if a given object is more like circle (both measures gets a high values) or ellipse (ellipticity measure has high value). By this knowledge we can observe/investigate, e.g., movement on the whole object by the simpler approach—circle or ellipse, respectively (compare with Fig. 1). we first use the red filter, and next fill the inside). Then we apply to the picture the affine operation which transforms this ellipse into a circle, which is given by the following formula x → −1/2(x − μC ). C This procedure transforms the elliptical object almost into a circle. For further example we refer the reader to [ 19 ]. 6.2 Noise Resistance 6.2.1 Shape Boundary Noise Figure 5 illustrates the robustness of E2. Presented shapes have similar measured ellipticity even though the last shape has a very high noise level. The noise is added to the shape boundary; thus, the perimeter of the object is increased. This experiment shows that the new measure can cope with such a situation. 6.2.2 Salt and Pepper In common applications the images which we are working with contain same noise—random, unwanted signal. Figure 13 illustrates the reliability of C2 for salt and pepper noise, for which a certain amount of the pixels in the image are either black or white. The percentage level describes the probability (a) EN (S) ∈ (0, 1] for all sets S; (b) CN (S) ∈ (0, 1] for all sets S; (c) EN (S) = 1 ⇔ S is an ellipse; (d) CN (S) = 1 ⇔ S is a N -ball(see footnote6); (e) CN is invariant with respect to similarity and isometric transformations; (f) EN is invariant with respect to affine transformations. Proof Items (a)–(d) follow directly form the Corollaries 1 and 2. Items (e) and (f) follows from the properties of the covariance matrix. In the following part of this section the new circularity measure properties are illustrated. When we apply the measure to image data, we shall threat a single pixel (point with coordinates (x , y) ∈ ([1, w] ∩ Z) × ([1, h] ∩ Z), where w × h is the size of the image) like a square. It allows to convert the discrete data of an image to continuous, which is more natural and consistent with the human perception of images. To calculate the measure of such transformation, we use the formulas from the Eq. (4), which is helpful for the calculation properties of an arbitrary set, namely the covariance matrix. Moreover, it is convenient to treat the discrete set as a set of hypercubes, then we can, in a more natural way, introduce the continuous densities. In the case of the image data we set δ = 1, which is because the coordinates of each pixel are integers. In 3-dimensional examples we use an arbitrary δ value. 6.1 Non-frontal View Image Correction A basic limitation of many image processing algorithm is that they require an “on-axis” image of investigated object. Clearly in most “real-life” pictures we have only side-view of the object. Figure 12 presents a basic concept how to deal with such situation. Namely, we modify the picture by respective affine transformation so that the elliptical object becomes circle-shaped. To do so we fit an optimal ellipse (denote it by C ) to the object we assume to be circular, in our case road sign (to obtain the shape of the road sign 6 With accuracy to the set of Lebesgue measure zero. Fig. 5 Measured ellipticity E2 of a shape with added noise [ 31 ] of occurrence of such kind of noise. The experiments shows that the covariance matrix which is base component enforces that C2 of plays well in such situations. 6.2.3 Missing Values Resistance The C2 is capable of handling unknown or missing values. Figure 14 presents the resistance of C2 for such kind of data. The percentage describes the overall level of unknown data. It is important to notice that the circularity value increases with the increase amount of present data. 6.3 Circle Estimation Figure 6 presents circularity measure C2 for regular polygons from equilateral triangle to dodecagon. The aim of this example is to find the good approximation for a circle. From Theorem 3 we derive that E2 reaches 1 only for a perfect circle. Thus, we want to acquire a simple template which can be treated as an approximation of a circle. First of all, we can confirm that the circularity measure behaves in a natural way—it increases with the number of polygon sides. Figure 6 shows that for a hexagon a value of 0.9924 is reached, which gives a two decimal places accuracy. Moreover, if a higher precision is needed, a decagon provides an accuracy of three decimal places. 6.4 E2 and C2 Behavior Figure 7 presents images ranked with respect to C2. Different rank is obtained by measures Cst and CH . This example illustrates how shape changing could lead to differences in the measured circularity. In this case the changes in the measured circularity C2 are in accordance with the natural perception of how a circularity measure should behave. Figure 8 presents the same experiment for E2. We highlight that the values of Cst and CH were taken from [ 31 ], while EH was taken from [ 30 ]. Moreover, the values of C2 and CH are theoretically equal—compare with Observation 1—the differences are caused by a numerical Fig. 6 Circularity measure for regular polygons from equilateral triangle to dodecagon. C2 of hexagon gives a one decimal place accuracy, while decagon gives an accuracy of three decimal places. Images in boxes represent the approximation of circle with an accuracy of two and three decimal places with respect to C2 measure Fig. 7 Shapes are ranked with respect to their measure C2 circularities [ 31 ]. Description under the images also contains Cst and CH (a) C2 = 0.8270 CH = 0.8265 Cst = 0.6046 (b) C2 = 0.8142 CH = 0.8174 Cst = 0.2335 (c) C2 = 0.7827 CH = 0.7851 Cst = 0.5772 (d) C2 = 0.7765 CH = 0.7777 Cst = 0.2677 (e) C2 = 0.7725 CH = 0.7789 Cst = 0.0651 Fig. 8 Shapes are ranked with respect to their E2 ellipticity measure [ 30 ]. Description under the images also contains EH and C2 (a) Fig. 10 Approximation of a set B = {(x, y, z) ∈ R3 approximation accuracy : x + 3y2 + 2z2 ≤ 1} by cubes with side δ. Ellipticity of set B increases with the 2 error. On the other hand, E2 and EH are in general not equal— see Fig. 8i. 6.5 3D Shapes In this experiment, 3-dimensional sets are considered and defined as follows A = {(x , y, z) ∈ R3 : x 2 + y2 + z2 ≤ 1}, B = {(x , y, z) ∈ R3 : x 2 + 3y2 + 2z2 ≤ 1}, C = {(x , y, z) ∈ R3 : 1 ≤ 21 x 2 + y2 + z2 ≤ 2 and x yz ≥ 0}. Figures 9a, 10a, and 11a illustrate the completed shapes of sets A, B, and C . The next step in this experiment is to approximate the shape by fixed size cubes. Let S denote the considered object. To obtain this approximation we proceed as follows: put Q = ∪x∈P Qx ; – we choose δ > 0; – by taking P = S ∩ (δZ)3 we obtain a discrete representation of our shape S; – each point x ∈ P is replaced by the cube of side δ. The center coordinates of such cube Qx is the same as replaced point namely μ(Qx ) = x for same x ∈ P. We (b) δ = 1.00 E2 = 0.6101 C2 = 0.5917 Fig. 12 Road sign image preprocessing. a Original road sign image. The middle image is a binarization of original image (b) according to values of its red channel. After the described operations, the shape of the sign was changed and is circle-like, see (c) Fig. 13 The C2 measure in case of images with salt and pepper noise Fig. 14 The C2 for the sets with unknown and missing values (a) original image (b) red channel binarization (c) image after preprocessing – we calculate the circularity and the ellipticity for obtained shape by equations from Corollaries 1 and 2 as follows: , . It is worth mentioning that the covariance matrix for each cube Q x is equal to 112 δ5I. Figures 9 and 10 present examples of a sphere and an ellipsoid, respectively. The ellipticity and circularity measure increases with the approximation accuracy. It can thus be concluded that the behavior of the new measure is natural even in higher dimensions. Figure 11 presents the situation for a shape with a hole. Both measures respond to this defect correctly and the calculated value is low (Figs. 12, 13, 14). 7 Conclusions The authors have placed their research efforts in the field of pattern recognition to establish a new measure of circularity and ellipticity based on moments. The proposed measure works in arbitrary dimensions, so we can test, e.g., for N-squares. The theoretical background and the proof that the conditions are well defined are also presented in this work. This approach can be treated as a generalization of the measures Cst and EI mentioned in Sect. 2. However, the authors’ approach can be applied in arbitrary dimensions (see Sect. 6). The fact that circles and ellipses maximize the above invariant has enabled the authors to introduce a new circularity CN and ellipticity EN measure defined in Corollaries 1 and 2. It is shown that CN and EN range over the interval (0, 1] and equal 1 if and only if the investigated set is, respectively, circle or ellipse. Experiments provided an illustration for the theoretical observations and demonstrate the applicability of the new ellipticity and circularity measures. The presented experiments emphasize the advantages of the new measures: – behavior consistent with the intuition; – invariance for similarity transformations; – applicability in higher dimensions; – allows a simpler description of any given object; – the calculation time is significantly reduced. Acknowledgments This work was supported by the Polish National Centre of Science Grant No. 2012/07/N/ST6/02192. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecomm ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Jacek Tabor received a master degree from mathematics at the Jagiellonian University, Kraków, Poland, in 1997. During the time period 1997–1998, he was on Fulbright Scholarship at the SUNY at Buffalo. In 2000, he obtained his Ph.D. in mathematics at the Jagiellonian University. Currently holds a full professor position at Faculty of Matematics and Computer Science at Jagiellonian University, Kraków, Poland. 1. Belongie , S. , Malik , J. , Puzicha , J.: Shape matching and object recognition using shape contexts . IEEE Trans. Pattern Anal. Mach. Intell . 24 ( 4 ), 509 - 522 ( 2002 ) 2. Bennett , C.H. , Gcs , P. , Li , M. , Vitnyi , P. , Zurek , W. : Information distance . IEEE Trans. Inform. Theory 44 ( 4 ), 1407 - 1423 ( 1998 ) 3. Bowman , E.T. , Soga , K. , Drummond , W. : Particle shape characterisation using fourier descriptor analysis . Geotechnique 51 ( 6 ), 545 - 554 ( 2001 ) 4. Chiang , C.-C., Ho , M.- C. , Liao , H.-S., Pratama , A. , Syu , W.-C.: Detecting and recognizing traffic lights by genetic approximate ellipse detection and spatial texture layouts . Int. J. Innov. Comput. Inf. Control 7 ( 12 ), 6919 - 6934 ( 2011 ) 5. Cover , T. , Thomas , J. : Elements of Information Theory . Wiley, New York ( 2006 ) 6. Cox , E.: A method of assigning numerical and percentage values to the degree of roundness of sand grains . J. Paleontol . 1 ( 3 ), 179 - 183 ( 1927 ) 7. Di Ruberto , C. , Dempster , A. : Circularity measures based on mathematical morphology . Electron. Lett . 36 ( 20 ), 1691 - 1693 ( 2000 ) 8. Fitzgibbon , A. , Pilu , M. , Fisher, R.: Direct least square fitting of ellipses . IEEE Trans. Pattern Anal. Mach. Intell . 21 ( 5 ), 476 - 480 ( 1999 ) 9. Flusser , J. , Suk , T. : Pattern recognition by affine moment invariants . Pattern Recognit . 26 ( 1 ), 167 - 174 ( 1993 ) 10. Flusser , J. , Zitova , B. , Suk , T. : Moments and Moment Invariants in Pattern Recognition . Wiley, New York ( 2009 ) 11. Liu , Meizhu , et al.: Shape retrieval using hierarchical total Bregman soft clustering . IEEE Trans. Pattern Anal. Mach. Intell . 34 ( 12 ), 2407 - 2419 ( 2012 ) 12. Gorelick , L. , Galun , M. , Sharon , E. , Basri , R. , Brandt , A. : Shape representation and classification using the poisson equation . IEEE Trans. Pattern Anal. Mach. Intell . 28 ( 12 ), 1991 - 2005 ( 2006 ) 13. Haralick , R.M.: A measure for circularity of digital figures . IEEE Trans. Syst. Man Cybern . 4 , 394 - 396 ( 1974 ) 14. Herrera-Navarro , A.M. , Jiménez-Hernández , H. , Terol-Villalobos , I.R.: A probabilistic measure of circularity . In: Combinatorial Image Analaysis , pp. 75 - 89 . Springer ( 2012 ) 15. Hu , M. : Visual pattern recognition by moment invariants . IRE Trans. Inf. Theory 8 ( 2 ), 179 - 187 ( 1962 ) 16. Ji , Q. : 3D face pose estimation and tracking from a monocular camera . Image Vision Comput . 20 ( 7 ), 499511 ( 2002 ) 17. Kullback , S. , Leibler , R.A. : On information and sufficiency . Ann. Math. Stat. 22 ( 1 ), 79 - 86 ( 1951 ) 18. Mahalanobis , P. : On the generalized distance in statistics . In: Proceedings of the National Institute of Sciences of India , Vol. 2 , pp. 49 - 55 . New Delhi ( 1936 ) 19. Misztal , K. , Tabor , J.: Mahalanobis distance-based algorithm for ellipse growing in iris preprocessing . In: Computer Information Systems and Industrial Management , pp. 158 - 167 . Springer, Berlin ( 2013 ) 20. OpenStax College (Wikipedia Commons): Illustration from Anatomy & Physiology, Connexions . http://cnx.org/content/ col11496/1.6/ (red channel). In Wikipedia https://commons. wikimedia.org/wiki/File:1911_ Sickle_Cells.jpg ( 2013 ). Accessed 19 June 2013 21. Peura , M. , Iivarinen , J.: Efficiency of simple shape descriptors . In: Aspects of Visual Form , pp. 443 - 451 ( 1997 ) 22. Proffitt , D. : The measurement of circularity and ellipticity on a digital grid . Pattern Recogn . 15 ( 5 ), 383 - 387 ( 1982 ) 23. Rahtu , E. , Salo , M. , Heikkila , J.: A new convexity measure based on a probabilistic interpretation of images . IEEE Trans. Pattern Anal. Mach. Intell . 28 ( 9 ), 1501 - 1512 ( 2006 ) 24. Rosin , P.L. : Measuring shape: ellipticity, rectangularity, and triangularity . Mach. Vis. Appl . 14 ( 3 ), 172 - 184 ( 2003 ) 25. Rosin , P.L. , Žunic ´ , J.: Measuring squareness and orientation of shapes . J. Math. Imaging Vis . 39 ( 1 ), 13 - 27 ( 2011 ) 26. Sonka , M. , Hlavac , V. , Boyle , R. , et al.: Image Processing, Analysis, and Machine Vision . PWS Publication, New York ( 1999 ) 27. Stojmenovic´, M. , Nayak , A. , Zunic , J.: Measuring linearity of planar point sets . Pattern Recogn . 41 ( 8 ), 2503 - 2511 ( 2008 ) 28. Tabor , J. , Spurek , P. : Cross-entropy clustering . Pattern Recogn . 47 ( 9 ), 3046 - 3059 ( 2014 ) 29. Zabrodsky , H. , Peleg , S. , Avnir , D. : Symmetry as a continuous feature . IEEE Trans. Pattern Anal. Mach. Intell . 17 ( 12 ), 1154 - 1166 ( 1995 ) 30. Žunic´, D. , Žunic´, J.: Shape ellipticity from Hu moment invariants . Appl. Math. Comput . 226 , 406 - 414 ( 2014 ) 31. Žunic´, J. , Hirota , K. , Rosin , P.L. : A Hu moment invariant as a shape circularity measure . Pattern Recogn . 43 ( 1 ), 47 - 57 ( 2010 ) Krzysztof Misztal received two master degrees from mathematics and applied computer science at the Jagiellonian University, Kraków, Poland, in 2009 and 2010 . In 2015, he obtained his Ph.D. in computer science at the Jagiellonian University. Currently he holds an assistant positions at Faculty of Matematics and Computer Science at Jagiellonian University, Kraków, Poland.

This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs10851-015-0618-4.pdf

Krzysztof Misztal, Jacek Tabor. Ellipticity and Circularity Measuring via Kullback–Leibler Divergence, Journal of Mathematical Imaging and Vision, 2016, 136-150, DOI: 10.1007/s10851-015-0618-4