Graph Regularized Nonnegative Matrix Factorization with Sparse Coding

Hindawi Publishing Corporation Mathematical Problems in Engineering Volume 2015, Article ID 239589, 11 pages http://dx.doi.org/10.1155/2015/239589 Research Article Graph Regularized Nonnegative Matrix Factorization with Sparse Coding Chuang Lin and Meng Pang School of Software, Dalian University of Technology, Dalian 116620, China Correspondence should be addressed to Chuang Lin; linchuang Received 13 January 2015; Revised 19 February 2015; Accepted 20 February 2015 Academic Editor: Nazrul Islam Copyright © 2015 C. Lin and M. Pang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. In this paper, we propose a sparseness constraint NMF method, named graph regularized matrix factorization with sparse coding (GRNMF SC). By combining manifold learning and sparse coding techniques together, GRNMF SC can efficiently extract the basic vectors from the data space, which preserves the intrinsic manifold structure and also the local features of original data. The target function of our method is easy to propose, while the solving procedures are really nontrivial; in the paper we gave the detailed derivation of solving the target function and also a strict proof of its convergence, which is a key contribution of the paper. Compared with sparseness constrained NMF and GNMF algorithms, GRNMF SC can learn much sparser representation of the data and can also preserve the geometrical structure of the data, which endow it with powerful discriminating ability. Furthermore, the GRNMF SC is generalized as supervised and unsupervised models to meet different demands. Experimental results demonstrate encouraging results of GRNMF SC on image recognition and clustering when comparing with the other state-of-the-art NMF methods. 1. Introduction Previous studies have shown that there is a psychological and physiological evidence for parts-based representation in the human brain [1–5]. NMF is such kind of parts-based matrix factorization methods, which can find out the local features from the original data in nonnegative sense. Indeed, the nonnegative constraint leads to a parts-based representation because it allows only additive combinations of components, not subtractive ones. Generally, NMF method is to find two nonnegative matrices whose product provides a good approximation to the original matrix, and, at the same time, it can also learn the parts of objects, which make them very important in some real applications, for example, in face recognition [6–9] and document clustering [10, 11] fields. However, the standard NMF [5, 6] algorithm has several limitations, which has been discussed extensively. One of the notable limitations of standard NMF is that it does not always result in completely parts-based representations. Researchers tried to solve this problem by incorporating sparseness constraints [12–14]. These approaches extended the NMF framework to include an adjustable sparseness parameter to learn more localized representation. However, previous sparseness constraint NMF approaches paid main attention to the sparseness property, while ignoring preserving the intrinsic geometric structure of the original data, which is vital for classification and clustering. Recent research shows that when the data is sampled from a probability distribution that resides in or nearby to a submanifold of the ambient space, manifold learning [15–19] can be used to preserve the intrinsic (geometrical) structure. In order to preserve the intrinsic structure of the original data, He and Cai proposed Graph Regularized NMF (GNMF) methods [20, 21], which incorporated local preserving projection (LPP) technique [22] to NMF framework. The experimental results showed that GNMF achieved higher recognition rates and better clustering effect in some popular facial databases (e.g., ORL and YALE database) comparing with previous sparseness constraint NMF [20, 21]. It means for some datasets, which have apparent geometrical structure, that GNMF really works. While, GNMF 2 Mathematical Problems in Engineering still has disadvantage, it cannot ensure the sparseness of the factorization results, which limited the discriminative ability and also increased the computational expense and memory space. Hence, we are motivated to combine the advantage of manifold and sparseness constraint and propose the GRNMF SC algorithm, which can not only preserve the geometrical structure, but also learn much sparser representations of the input data. It needs to emphasize that it is nontrivial to solve the objective function which simultaneously incorporates the Laplacian regularization and sparseness constraint into the NMF framework, because the sophisticated L1-norm solving tools cannot be adopted directly. We start from the initial idea of designing GNMF and incorporate the sparseness constraint smoothly. The concrete steps are first, construct a convex objection function by imposing the above constrains and then develop an optimization algorithm with multiplicative update rules to minimize this objective function. Finally, prove the algorithm can converge to a local minimum. Furthermore, we extend GRNMF SC to both supervised (S-GRNMF SC) and unsupervised versions (GRNMF SC) for image recognition and clustering, respectively; in clustering, the class labels are not available. Experimental results demonstrate that supervised GRNMF SC achieved higher recognition rates, especially in occluded face recognition, when comparing with the typical sparseness based NMF and manifold based NMF methods, and the unsupervised GRNMF SC obtained better clustering performance comparing with the popular clustering algorithms. The rest of the paper is organized as follows. In Section 2, a brief review of standard NMF and its typical sparse variants is given. In Section 3, the proposed GRNMF SC method and a proof of its convergence are given. Experimental results on image recognition and clustering are presented in Section 4. We conclude the paper and plan the future work in Section 5. 2. Reviews of Standard NMF and Its Sparse Variants In this section, we briefly describe the standard NMF algorithm [5] and two typical sparseness constraint NMF algorithms [8, 13]; the reason of introducing the two sparseness constraint NMF algorithms is that our method is inspired by them. The introduction of GNMF is merged with GRNMF SC in Section 3. 2.1. Nonnegative Matrix Factorization (NMF). First, standard NMF [5, 6] is introduced. Given a data matrix X = [x1 , . . . , x𝑛 ] ∈ R𝑚×𝑛 , each column of X is an 𝑚-dimensionalsample vector with nonnegative values. NMF aims to find two nonnegative matrices U ∈ R𝑚×𝑟 and V ∈ R𝑚×𝑟 whose product can well approximate the original matrix. That is, it is to minimize the following cost function: 󵄩 󵄩2 𝑂 = 󵄩󵄩󵄩󵄩X − UV𝑇 󵄩󵄩󵄩󵄩𝐹 . (1) 𝐹 represents the Frobenius norm. (...truncated)