Convergence of Linear Bregman ADMM for Nonconvex and Nonsmooth Problems with Nonseparable Structure

Hindawi Complexity Volume 2020, Article ID 6237942, 14 pages https://doi.org/10.1155/2020/6237942 Research Article Convergence of Linear Bregman ADMM for Nonconvex and Nonsmooth Problems with Nonseparable Structure Miantao Chao ,1 Zhao Deng,1 and Jinbao Jian 1 2 2 Department of Mathematics, Guangxi University, Nanning 530004, China Guangxi University for Nationalities, Nanning, China Correspondence should be addressed to Jinbao Jian; Received 5 September 2019; Revised 6 January 2020; Accepted 27 January 2020; Published 26 February 2020 Academic Editor: Zhile Yang Copyright © 2020 Miantao Chao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The alternating direction method of multipliers (ADMM) is an effective method for solving two-block separable convex problems and its convergence is well understood. When either the involved number of blocks is more than two, or there is a nonconvex function, or there is a nonseparable structure, ADMM or its directly extend version may not converge. In this paper, we proposed an ADMM-based algorithm for nonconvex multiblock optimization problems with a nonseparable structure. We show that any cluster point of the iterative sequence generated by the proposed algorithm is a critical point, under mild condition. Furthermore, we establish the strong convergence of the whole sequence, under the condition that the potential function satisfies the Kurdyka–Łojasiewicz property. This provides the theoretical basis for the application of the proposed ADMM in the practice. Finally, we give some preliminary numerical results to show the effectiveness of the proposed algorithm. 1. Introduction In this paper, we consider the following possibly nonconvex and nonsmooth optimization problem: N− 1 min 􏽐 fi xi 􏼁 + g x1 , · · · , xN− 1 , y􏼁, i�1 N− 1 s.t. (1) 􏽐 Ai xi + By � b, i�1 where xi ∈ Rni (i � 1, 2, . . . , N − 1), y ∈ Rn are variables vectors, g: Rl ⟶ R(l � n1 + n2 + · · · + nN− 1 + n) is differentiable, and each fi : Rni ⟶ R ∪ {+∞} is proper and lower semicontinuous, Ai ∈ Rm×ni (i � 1, 2, . . . , N − 1), B ∈ Rm×n are given matrix, and b ∈ Rm . The alternating direction method of multipliers (ADMM) is a very effective method for solving the convex two-block optimization problem [1, 2]. A natural idea is to extend ADMM to solve problem (1). However, ADMM or its directly extend version may not converge, when either the involved number of blocks is more than two, or there is a nonconvex function, or there is a nonseparable structure. Recently, there have been a few developments on it, e.g., [3–13]. Hong et al. [6] considered the sharing and consensus problem and showed that the classical ADMM converges to the set of stationary solutions, only provided that the penalty parameter in the augmented Lagrangian is chosen to be sufficiently large. Li and Pong [8] studied the convergence of ADMM for some special two-block nonconvex models, where one of the matrices A and B is an identity matrix. Wang et al. [9, 10] studied the convergence of the nonconvex Bregman ADMM algorithm, which includes ADMM as a special case. Wang et al. [11] studied the convergence of the ADMM for nonconvex nonsmooth optimization with a nonseparable structure. Guo et al. [4, 5] studied the convergence of classical ADMM for two-block and multiblock nonconvex models where one of the matrices is an identity matrix. Yang et al. [13] studied the convergence of the ADMM for a nonconvex optimization model which come from the background/foreground extraction. The purpose and the main contribution of this paper is to propose and prove the convergence of a new variant ADMM 2 Complexity for nonconvex coupled problems (1). The novelty of this paper can be summarized as follows: (1) Compared to the existing literature, the model in this paper is more general. There is no nonseparable structure in the models considered by [4–10, 12, 13]. Wang et al. [11] considered two scenarios. If g(x1 , . . . , xN− 1 , y) � g1 (x1 , . . . , xN− 1 ) + h(y), then (1) is the scenario 1 in [11]. If fi (x) ≡ 0, for i � 1, 2, . . . , N − 1, then (1) becomes the scenario 2 in [11]. Furthermore, in this paper, the matrices Ai (i � 1, 2, . . . , N − 1) and B are possibly not full column or row rank. (2) The proposed algorithm combines linearization technology with regularization technology. Linearization technology and regularization technology can effectively reduce the difficulty of the solving subproblems. The rest of this paper is organized as follows. In Section 2, some basic concepts and necessary preliminaries for further analysis are summarized. In Section 3, we propose the algorithm and analyze the convergence of it for 3-block nonconvex and nonsmooth coupled problems. Finally, some conclusions are made in Section 4. Δϕ (x, y) � ϕ(x) − ϕ(y) − < ∇ϕ(y), x − y > , (3) for all x, y ∈ Rn . The Bregman distance plays an important role in iterative algorithms. The Bregman distance share many similar nice properties of the Euclidean distance. However, the Bregman distance is not a metric, since it does not satisfy the triangle inequality nor symmetry. Some examples of Bregman distance include [16] (i) Classical Euclidean distance: if ϕ(x) � ‖x‖2 , then Δϕ (x, y) � ‖x − y‖2 (ii) Itakura–Saito distance: if ϕ(x) � 􏽐m i�1 xi (log xi ), m then Δϕ (x, y) � 􏽐m x (log(x /y )) − 􏽐 i i i�1 i i�1 (xi − yi ) 2 (iii) Mahalanobis distance: if ϕ(x) � ‖x‖Q � xT Qx with Q a symmetric positive definite matrix, then Δϕ (x, y) � ‖x − y‖2Q Let us now collect some useful properties about Bregman distance. Proposition 1 (see [15]). Let ϕ be differentiable and strongly convex function with modulus δ, then (i) Δϕ (x, y) ≥ 0 and Δϕ (x, y) � 0 if and only if x � y (ii) Δϕ (x, y) ≥ (δ/2)‖x − y‖2 for all x and y The following notations and definitions are quite standard and can be founded in [14, 17]. 2. Preliminaries Rn denotes the n-dimensional Euclidean space, R ∪ {+∞} denotes the extended real number set, and N denotes the natural number set. The image space of a matrix Q ∈ Rm×n is defined as Im Q: � {Qx: x ∈ Rn }. PQ (·) denotes the Euclidean projection onto Im Q. If matrix Q ≠ 0, let μQ denote the smallest positive singular value of the matrix QQT . ‖ · ‖ represents the Euclidean norm. dom(f): � {x ∈ Rn : f(x) < +∞} is the domain of a function f: Rn ⟶ R ∪ {+∞}. 􏼊x, y􏼋 � xT y � 􏽐ni�1 xi yi . [η1 < f < η2 ]: � {x: η1 < f(x) < η2 }. For a set S ⊂ Rn and a point x ∈ Rn , let d(x, S) � inf ‖y − x‖2 . If S � ∅, we set d(x, S) � +∞ for all y∈S x ∈ Rn . For a point-to-set mapping F, its graph is defined by Graph F: � 􏼈(x, y): y ∈ F(x)􏼉. Definition 3. Let f: Rn ⟶ R ∪ {+∞} be a proper lower semicontinuous function. (i) The Frėchet subdifferential, or regular subdifferential, of f at x ∈ dom f is f(y) − f(x) − < x∗ , y − x > 􏽢 zf(x) � 􏼨x∗ : lim inf ≥ 0􏼩. y≠x y⟶x ‖y − x‖ (...truncated)