Electronic Journal of Statistics

Multivariate sharp quadratic bounds via $\mathbf{\Sigma}$-strong convexity and the Fenchel connection

Ryan P. Browne and Paul D. McNicholas

Full-text: Open access


Sharp majorization is extended to the multivariate case. To achieve this, the notions of $\sigma$-strong convexity, monotonicity, and one-sided Lipschitz continuity are extended to $\mathbf{\Sigma}$-strong convexity, monotonicity, and Lipschitz continuity, respectively. The connection between a convex function and its Fenchel-Legendre transform is then developed. Sharp majorization is illustrated in single and multiple dimensions, and we show that these extensions yield improvements on bounds given within the literature. The new methodology introduced herein is used to develop a variational approximation for the Bayesian multinomial regression model.

Article information

Electron. J. Statist., Volume 9, Number 2 (2015), 1913-1938.

Received: May 2014
First available in Project Euclid: 27 August 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Browne, Ryan P.; McNicholas, Paul D. Multivariate sharp quadratic bounds via $\mathbf{\Sigma}$-strong convexity and the Fenchel connection. Electron. J. Statist. 9 (2015), no. 2, 1913--1938. doi:10.1214/15-EJS1061. https://projecteuclid.org/euclid.ejs/1440680331

Export citation


  • [1] Ashford, J. R. (1959). An approach to the analysis of data for semiquantal respsones in biological assay., Biometrics 15, 573–581.
  • [2] Bartholomew, D. J. and Knott, M. (1999). Latent variable models and factor analysis. In, Kendall’s Library of Statistics (2nd ed.), Volume 7. London: Edward Arnold.
  • [3] Beal, M. J. (2003)., Variational Algorithms for Approximate Bayesian Inference. Ph.D. Thesis, Gatsby Computational Neuroscience Unit, University College London.
  • [4] Böhning, D. (1992). Multinomial logistic regression algorithm., Annals of the Institute of Statistical Mathematics 44(1), 197–200.
  • [5] Böhning, D. and Lindsay, B. G. (1988). Monotonicity of quadratic-approximation algorithms., Annals of the Institute of Statistical Mathematics 40(4), 641–663.
  • [6] Corduneanu, A. and Bishop, C. (2001). Variational Bayesian model selection for mixture distributions., Artificial Intelligence and Statistics 37, 27–34.
  • [7] de Leeuw, J. and Lange, K. (2009). Sharp quadratic majorization in one dimension., Computational Statistics and Data Analysis 53(7), 2471–2484.
  • [8] Dempster, A., Laird, N., and Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm., Journal of the Royal Statistical Society: Series B 38, 1–38.
  • [9] Donchev, T. and Farkhi, E. (1998). Stability and Euler approximation of one-sided Lipschitz differential inclusions., SIAM Journal on Control and Optimization 36(2), 780–796.
  • [10] Heiser, W. J. (1995). Convergent computation by iterative majorization: Theory and applications in multidimensional data analysis. In: Krzanowski, W. J. (Ed.), Recent Advances in Descriptive Multivariate Analysis, Volume 58. Oxford: Springer-Verlag.
  • [11] Hunter, D. R. and Lange, K. (2004). A tutorial on MM algorithms., The American Statistician 58, 30–37.
  • [12] Jaakkola, T. S. W. and Jordan, M. I. W. (2000). Bayesian parameter estimation via variational methods., Statistics and Computing 10, 25–37.
  • [13] Jordan, M., Ghahramani, Z., Jaakkola, T., and Saul, L. (1999). An introduction to variational methods for graphical models., Machine Learning 37, 183–223.
  • [14] Lange, K., Hunter, D. R., and Yang, I. (2000). Optimization transfer using surrogate objective functions (with discussion)., Journal of Computational and Graphical Statistics 9, 1–59.
  • [15] McCullagh, P. and Nelder, J. A. (1989)., Generalized Linear Models (2nd ed.), Volume 2. London: Chapman & Hall.
  • [16] McGrory, C. A. and Titterington, D. (2007). Variational approximations in Bayesian model selection for finite mixture distributions., Computational Statistics and Data Analysis 51, 5352–5367.
  • [17] Rockafellar, R. T. and Wets, R. J.-B. (2009)., Variational Analysis. New York: Springer-Verlag.
  • [18] Subedi, S. and McNicholas, P. D. (2007). Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions., Advances in Data Analysis and Classifcation 8(2), 167–193.
  • [19] Teschendorff, A., Wang, Y., Barbosa-Morais, N., Brenton, J., and Caldas, C., A variational Bayesian mixture modelling framework for cluster analysis of gene-expression data., Bioinformatics 21.
  • [20] Tipping, M. (1999). Probabilistic visualisation of high-dimensional binary data. In M. Kearns, S. Solla, and D. Cohn (Eds.), Advances in Neural Information Processing Systems 11, Volume 11. Cambridge, MA, USA: MIT PRESS, pp. 592–598.
  • [21] Waterhouse, S., MacKay, D., and Robinson, T. (1996). Bayesian methods for mixture of experts., Advances in Neural Information Processing Systems 8.