Abstract and Applied Analysis

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

Caihua Chen, Yuan Shen, and Yanfei You

Full-text: Open access


We consider a class of linearly constrained separable convex programming problems whose objective functions are the sum of three convex functions without coupled variables. For those problems, Han and Yuan (2012) have shown that the sequence generated by the alternating direction method of multipliers (ADMM) with three blocks converges globally to their KKT points under some technical conditions. In this paper, a new proof of this result is found under new conditions which are much weaker than Han and Yuan’s assumptions. Moreover, in order to accelerate the ADMM with three blocks, we also propose a relaxed ADMM involving an additional computation of optimal step size and establish its global convergence under mild conditions.

Article information

Abstr. Appl. Anal., Volume 2013, Special Issue (2013), Article ID 183961, 7 pages.

First available in Project Euclid: 26 February 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Chen, Caihua; Shen, Yuan; You, Yanfei. On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks. Abstr. Appl. Anal. 2013, Special Issue (2013), Article ID 183961, 7 pages. doi:10.1155/2013/183961. https://projecteuclid.org/euclid.aaa/1393450485

Export citation


  • D. Gabay and B. Mercier, “A dual algorithm for the solution of nonlinear variational problems via finite element approximations,” Computational Mathematics with Applications, vol. 2, pp. 17–40, 1976.
  • R. Glowinski and A. Marrocco, “Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité, d'une classe de problèmes de Dirichlet non linéaires,” vol. 9, no. R-2, pp. 41–76, 1975.
  • J. Eckstein and D. Bertsekas, “On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators,” Mathematical Programming, vol. 55, no. 3, pp. 293–318, 1992.
  • D. Gabay, “Chapter ix applications of the method of multipliers to variational inequalities,” Studies in Mathematics and Its Applications, vol. 15, pp. 299–331, 1983.
  • R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer, New York, NY, USA, 1984.
  • R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, vol. 9, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa, USA, 1989.
  • B. He, L. Liao, D. Han, and H. Yang, “A new inexact alternating directions method for monotone variational inequalities,” Mathematical Programming, vol. 92, no. 1, pp. 103–118, 2002.
  • B. He and X. Yuan, “On the $O(1/n)$ convergence rate of the Douglas-Rachford alternating direction method,” SIAM Journal on Numerical Analysis, vol. 50, no. 2, pp. 700–709, 2012.
  • C. Chen, B. He, and X. Yuan, “Matrix completion via an alternating direction method,” IMA Journal of Numerical Analysis, vol. 32, no. 1, pp. 227–245, 2012.
  • M. Fazel, T. K. Pong, D. F. Sun, and P. Tseng, “Hankel matrix rank minimization with applications to system identification and realization,” SIAM Journal on Matrix Analysis and Applications, vol. 34, no. 3, pp. 946–977, 2012.
  • B. He, M. Xu, and X. Yuan, “Solving large-scale least squares semidefinite programming by alternating direction methods,” SIAM Journal on Matrix Analysis and Applications, vol. 32, no. 1, pp. 136–152, 2011.
  • J. Yang, Y. Zhang, and W. Yin, “A fast alternating direction method for tvl1-l2 signal reconstruction from partial fourier data,” IEEE Journal of Selected Topics in Signal Processing, vol. 4, no. 2, pp. 288–297, 2010.
  • J. Yang and Y. Zhang, “Alternating direction method algorithms for l1-problems in compressive sensing,” SIAM Journal on Scientific Computing, vol. 33, no. 1, pp. 250–278, 2011.
  • Z. Wen, D. Goldfarb, and W. Yin, “Alternating direction augmented Lagrangian methods for semidefinite programming,” Mathematical Programming Computation, vol. 2, no. 3-4, pp. 203–230, 2010.
  • M. Tao and X. Yuan, “Recovering low-rank and sparse components of matrices from incomplete and noisy observations,” SIAM Journal on Optimization, vol. 21, no. 1, pp. 57–81, 2011.
  • J. Yang, D. Sun, and K. Toh, “A proximal point algorithm for logdeterminant optimization with group Lasso regularization,” SIAM Journal on Optimization, vol. 23, no. 2, pp. 857–293, 2013.
  • Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, “RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 763–770, 2010.
  • K. Mohan, P. London, M. Fazel, D. Witten, and S. I. Lee, “Node-based learning of multiple gaussian graphical models,” http://arxiv.org/abs/1303.5145.
  • B. He, M. Tao, M. H. Xu, and X. M. Yuan, “Alternating directions based contraction method for generally separable linearly constrained convex programming problems,” Optimization, vol. 62, no. 4, pp. 573–596, 2013.
  • B. He, M. Tao, and X. Yuan, “Alternating direction method with Gaussian back substitution for separable convex programming,” SIAM Journal on Optimization, vol. 22, no. 2, pp. 313–340, 2012.
  • M. Hong and Z. Luo, “On the linear convergence of the alternating čommentComment on ref. [15?]: Please update the information of this reference [16?], if possible. direction Method of multipliers,” http://arxiv.org/abs/1208.3922.
  • D. Han and X. Yuan, “A note on the alternating direction method of multipliers,” Journal of Optimization Theory and Applications, vol. 155, no. 1, pp. 227–238, 2012.
  • R. Rockafellar, “Monotone operators and the proximal point algorithm,” SIAM Journal on Control and Optimization, vol. 14, no. 5, pp. 877–898, 1976.
  • R. Rockafellar, Convex Analysis, Princeton Mathematical Series, no. 28, Princeton University Press, Princeton, NJ, USA, 1970.
  • C. Ye and X.-M. Yuan, “A descent method for structured monotone variational inequalities,” Optimization Methods & Software, vol. 22, no. 2, pp. 329–338, 2007.