Bayesian Analysis

Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions

Yulai Cong, Bo Chen, and Mingyuan Zhou

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

We introduce a fast and easy-to-implement simulation algorithm for a multivariate normal distribution truncated on the intersection of a set of hyperplanes, and further generalize it to efficiently simulate random variables from a multivariate normal distribution whose covariance (precision) matrix can be decomposed as a positive-definite matrix minus (plus) a low-rank symmetric matrix. Example results illustrate the correctness and efficiency of the proposed simulation algorithms.

Article information

Source
Bayesian Anal. (2017), 21 pages.

Dates
First available in Project Euclid: 1 March 2017

Permanent link to this document
http://projecteuclid.org/euclid.ba/1488337478

Digital Object Identifier
doi:10.1214/17-BA1052

Keywords
Cholesky decomposition conditional distribution equality constraints high-dimensional regression structured covariance/precision matrix

Rights
Creative Commons Attribution 4.0 International License.

Citation

Cong, Yulai; Chen, Bo; Zhou, Mingyuan. Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions. Bayesian Anal., advance publication, 1 March 2017. doi: 10.1214/17-BA1052. http://projecteuclid.org/euclid.ba/1488337478


Export citation

References

  • Albert, J. H. and Chib, S. (1993). “Bayesian analysis of binary and polychotomous response data.” Journal of the American Statistical Association, 88(422): 669–679.
  • Altmann, Y., McLaughlin, S., and Dobigeon, N. (2014). “Sampling from a multivariate Gaussian distribution truncated on a simplex: a review.” In 2014 IEEE Workshop on Statistical Signal Processing (SSP), 113–116. IEEE.
  • Bazot, C., Dobigeon, N., Tourneret, J.-Y., Zaas, A. K., Ginsburg, G. S., and Hero III, A. O. (2013). “Unsupervised Bayesian linear unmixing of gene expression microarrays.” BMC Bioinformatics, 14(1): 1.
  • Bhattacharya, A., Chakraborty, A., and Mallick, B. K. (2016). “Fast sampling with Gaussian scale mixture priors in high-dimensional regression.” Biometrika, 103(4): 985.
  • Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). “Latent Dirichlet allocation.” Journal of Machine Learning Research, 3: 993–1022.
  • Botev, Z. (2016). “The normal law under linear restrictions: simulation and estimation via minimax tilting.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).
  • Caron, F. and Doucet, A. (2008). “Sparse Bayesian nonparametric regression.” In ICML, 88–95. ACM.
  • Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97(2): 465–480.
  • Chopin, N. (2011). “Fast simulation of truncated Gaussian distributions.” Statistics and Computing, 21(2): 275–288.
  • Cong, Y., Chen, B., and Zhou, M. (2017). “Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions: Supplementary Material.” Bayesian Analysis.
  • Cong, Y., Chen, B., and Zhou, M. (2017). “Deep latent Dirichlet allocation with topic-layer-adaptive stochastic gradient Riemannian (TLASGR) MCMC.” Preprint.
  • Damien, P. and Walker, S. G. (2001). “Sampling truncated normal, beta, and gamma densities.” Journal of Computational and Graphical Statistics, 10(2): 206–215.
  • Dobigeon, N., Moussaoui, S., Coulon, M., Tourneret, J.-Y., and Hero, A. O. (2009a). “Joint Bayesian endmember extraction and linear unmixing for hyperspectral imagery.” IEEE Transactions on Signal Processing, 57(11): 4355–4368.
  • Dobigeon, N., Moussaoui, S., Tourneret, J.-Y., and Carteret, C. (2009b). “Bayesian separation of spectral sources under non-negativity and full additivity constraints.” Signal Processing, 89(12): 2657–2669.
  • Doucet, A. (2010). “A note on efficient conditional simulation of Gaussian distributions.” Departments of Computer Science and Statistics, University of British Columbia.
  • Gelfand, A. E., Smith, A. F., and Lee, T.-M. (1992). “Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling.” Journal of the American Statistical Association, 87(418): 523–532.
  • Gelfand, A. E. and Smith, A. F. M. (1990). “Sampling-based approaches to calculating marginal densities.” Journal of the American Statistical Association, 85(410): 398–409.
  • Geman, S. and Geman, D. (1984). “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 721–741.
  • Geweke, J. (1991). “Efficient simulation from the multivariate normal and student-t distributions subject to linear constraints and the evaluation of constraint probabilities.” In Computing science and statistics: Proceedings of the 23rd symposium on the interface, 571–578.
  • Geweke, J. F. (1996). “Bayesian inference for linear models subject to linear inequality constraints.” In Modelling and Prediction Honoring Seymour Geisser, 248–263. Springer.
  • Golub, G. H. and Van Loan, C. F. (2012). Matrix Computations, volume 3. JHU Press.
  • Heckerman, D. (1998). “A tutorial on learning with Bayesian networks.” In Learning in graphical models, 301–354. Springer.
  • Hoffman, M., Blei, D., and Bach, F. (2010). “Online learning for latent Dirichlet allocation.” In NIPS.
  • Hoffman, Y. and Ribak, E. (1991). “Constrained realizations of Gaussian fields-A simple algorithm.” The Astrophysical Journal, 380: L5–L8.
  • Holmes, C. C. and Held, L. (2006). “Bayesian auxiliary variable models for binary and multinomial regression.” Bayesian Analysis, 1(1): 145–168.
  • Imai, K. and van Dyk, D. A. (2005). “A Bayesian analysis of the multinomial probit model using marginal data augmentation.” Journal of Econometrics, 124(2): 311–334.
  • Johndrow, J., Dunson, D., and Lum, K. (2013). “Diagonal orthant multinomial probit models.” In AISTATS, 29–38.
  • Lan, S., Zhou, B., and Shahbaba, B. (2014). “Spherical Hamiltonian Monte Carlo for constrained target distributions.” In ICML, 629–637.
  • Ma, Y., Chen, T., and Fox, E. (2015). “A complete recipe for stochastic gradient MCMC.” In NIPS, 2899–2907.
  • McCulloch, R. E., Polson, N. G., and Rossi, P. E. (2000). “A Bayesian analysis of the multinomial probit model with fully identified parameters.” Journal of Econometrics, 99(1): 173–193.
  • Neelon, B. and Dunson, D. B. (2004). “Bayesian isotonic regression and trend analysis.” Biometrics, 60(2): 398–406.
  • Pakman, A. and Paninski, L. (2014). “Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians.” Journal of Computational and Graphical Statistics, 23(2): 518–542.
  • Polson, N. G., Scott, J. G., and Windle, J. (2014). “The Bayesian bridge.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(4): 713–733.
  • Pritchard, J. K., Stephens, M., and Donnelly, P. (2000). “Inference of population structure using multilocus genotype data.” Genetics, 155(2): 945–959.
  • Robert, C. P. (1995). “Simulation of truncated normal variables.” Statistics and Computing, 5(2): 121–125.
  • Rodriguez-Yam, G., Davis, R. A., and Scharf, L. L. (2004). “Efficient Gibbs sampling of truncated multivariate normal with application to constrained linear regression.” Technical report.
  • Rue, H. (2001). “Fast sampling of Gaussian Markov random fields.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(2): 325–338.
  • Schmidt, M. (2009). “Linearly constrained Bayesian matrix factorization for blind source separation.” In NIPS, 1624–1632.
  • Tong, Y. L. (2012). The Multivariate Normal Distribution. Springer Science & Business Media.
  • Train, K. E. (2009). Discrete Choice Methods With Simulation. Cambridge University Press.
  • Zhou, M., Cong, Y., and Chen, B. (2016). “Augmentable gamma belief networks.” Journal of Machine Learning Research, 17(163): 1–44.
  • Zhou, M., Hannah, L., Dunson, D. B., and Carin, L. (2012). “Beta-negative binomial process and Poisson factor analysis.” In AISTATS, 1462–1471.

Supplemental materials