## Bayesian Analysis

- Bayesian Anal. (2017), 21 pages.

### Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions

Yulai Cong, Bo Chen, and Mingyuan Zhou

#### Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

**Full-text: Open access**

#### Abstract

We introduce a fast and easy-to-implement simulation algorithm for a multivariate normal distribution truncated on the intersection of a set of hyperplanes, and further generalize it to efficiently simulate random variables from a multivariate normal distribution whose covariance (precision) matrix can be decomposed as a positive-definite matrix minus (plus) a low-rank symmetric matrix. Example results illustrate the correctness and efficiency of the proposed simulation algorithms.

#### Article information

**Source**

Bayesian Anal. (2017), 21 pages.

**Dates**

First available in Project Euclid: 1 March 2017

**Permanent link to this document**

https://projecteuclid.org/euclid.ba/1488337478

**Digital Object Identifier**

doi:10.1214/17-BA1052

**Keywords**

Cholesky decomposition conditional distribution equality constraints high-dimensional regression structured covariance/precision matrix

**Rights**

Creative Commons Attribution 4.0 International License.

#### Citation

Cong, Yulai; Chen, Bo; Zhou, Mingyuan. Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions. Bayesian Anal., advance publication, 1 March 2017. doi: 10.1214/17-BA1052. https://projecteuclid.org/euclid.ba/1488337478

#### References

- Albert, J. H. and Chib, S. (1993). “Bayesian analysis of binary and polychotomous response data.”
*Journal of the American Statistical Association*, 88(422): 669–679. - Altmann, Y., McLaughlin, S., and Dobigeon, N. (2014). “Sampling from a multivariate Gaussian distribution truncated on a simplex: a review.” In
*2014 IEEE Workshop on Statistical Signal Processing (SSP)*, 113–116. IEEE. - Bazot, C., Dobigeon, N., Tourneret, J.-Y., Zaas, A. K., Ginsburg, G. S., and Hero III, A. O. (2013). “Unsupervised Bayesian linear unmixing of gene expression microarrays.”
*BMC Bioinformatics*, 14(1): 1. - Bhattacharya, A., Chakraborty, A., and Mallick, B. K. (2016). “Fast sampling with Gaussian scale mixture priors in high-dimensional regression.”
*Biometrika*, 103(4): 985. - Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). “Latent Dirichlet allocation.”
*Journal of Machine Learning Research*, 3: 993–1022. - Botev, Z. (2016). “The normal law under linear restrictions: simulation and estimation via minimax tilting.”
*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*. - Caron, F. and Doucet, A. (2008). “Sparse Bayesian nonparametric regression.” In
*ICML*, 88–95. ACM. - Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.”
*Biometrika*, 97(2): 465–480. - Chopin, N. (2011). “Fast simulation of truncated Gaussian distributions.”
*Statistics and Computing*, 21(2): 275–288. - Cong, Y., Chen, B., and Zhou, M. (2017). “Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions: Supplementary Material.”
*Bayesian Analysis*. - Cong, Y., Chen, B., and Zhou, M. (2017). “Deep latent Dirichlet allocation with topic-layer-adaptive stochastic gradient Riemannian (TLASGR) MCMC.”
*Preprint*. - Damien, P. and Walker, S. G. (2001). “Sampling truncated normal, beta, and gamma densities.”
*Journal of Computational and Graphical Statistics*, 10(2): 206–215. - Dobigeon, N., Moussaoui, S., Coulon, M., Tourneret, J.-Y., and Hero, A. O. (2009a). “Joint Bayesian endmember extraction and linear unmixing for hyperspectral imagery.”
*IEEE Transactions on Signal Processing*, 57(11): 4355–4368. - Dobigeon, N., Moussaoui, S., Tourneret, J.-Y., and Carteret, C. (2009b). “Bayesian separation of spectral sources under non-negativity and full additivity constraints.”
*Signal Processing*, 89(12): 2657–2669. - Doucet, A. (2010). “A note on efficient conditional simulation of Gaussian distributions.”
*Departments of Computer Science and Statistics, University of British Columbia*. - Gelfand, A. E., Smith, A. F., and Lee, T.-M. (1992). “Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling.”
*Journal of the American Statistical Association*, 87(418): 523–532. - Gelfand, A. E. and Smith, A. F. M. (1990). “Sampling-based approaches to calculating marginal densities.”
*Journal of the American Statistical Association*, 85(410): 398–409. - Geman, S. and Geman, D. (1984). “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.”
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 721–741. - Geweke, J. (1991). “Efficient simulation from the multivariate normal and student-t distributions subject to linear constraints and the evaluation of constraint probabilities.” In
*Computing science and statistics: Proceedings of the 23rd symposium on the interface*, 571–578. - Geweke, J. F. (1996). “Bayesian inference for linear models subject to linear inequality constraints.” In
*Modelling and Prediction Honoring Seymour Geisser*, 248–263. Springer. - Golub, G. H. and Van Loan, C. F. (2012).
*Matrix Computations*, volume 3. JHU Press. - Heckerman, D. (1998). “A tutorial on learning with Bayesian networks.” In
*Learning in graphical models*, 301–354. Springer. - Hoffman, M., Blei, D., and Bach, F. (2010). “Online learning for latent Dirichlet allocation.” In
*NIPS*. - Hoffman, Y. and Ribak, E. (1991). “Constrained realizations of Gaussian fields-A simple algorithm.”
*The Astrophysical Journal*, 380: L5–L8. - Holmes, C. C. and Held, L. (2006). “Bayesian auxiliary variable models for binary and multinomial regression.”
*Bayesian Analysis*, 1(1): 145–168.Mathematical Reviews (MathSciNet): MR2227368

Digital Object Identifier: doi:10.1214/06-BA105

Project Euclid: euclid.ba/1340371078 - Imai, K. and van Dyk, D. A. (2005). “A Bayesian analysis of the multinomial probit model using marginal data augmentation.”
*Journal of Econometrics*, 124(2): 311–334. - Johndrow, J., Dunson, D., and Lum, K. (2013). “Diagonal orthant multinomial probit models.” In
*AISTATS*, 29–38. - Lan, S., Zhou, B., and Shahbaba, B. (2014). “Spherical Hamiltonian Monte Carlo for constrained target distributions.” In
*ICML*, 629–637. - Ma, Y., Chen, T., and Fox, E. (2015). “A complete recipe for stochastic gradient MCMC.” In
*NIPS*, 2899–2907. - McCulloch, R. E., Polson, N. G., and Rossi, P. E. (2000). “A Bayesian analysis of the multinomial probit model with fully identified parameters.”
*Journal of Econometrics*, 99(1): 173–193. - Neelon, B. and Dunson, D. B. (2004). “Bayesian isotonic regression and trend analysis.”
*Biometrics*, 60(2): 398–406.Mathematical Reviews (MathSciNet): MR2066274

Digital Object Identifier: doi:10.1111/j.0006-341X.2004.00184.x - Pakman, A. and Paninski, L. (2014). “Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians.”
*Journal of Computational and Graphical Statistics*, 23(2): 518–542. - Polson, N. G., Scott, J. G., and Windle, J. (2014). “The Bayesian bridge.”
*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*, 76(4): 713–733. - Pritchard, J. K., Stephens, M., and Donnelly, P. (2000). “Inference of population structure using multilocus genotype data.”
*Genetics*, 155(2): 945–959. - Robert, C. P. (1995). “Simulation of truncated normal variables.”
*Statistics and Computing*, 5(2): 121–125. - Rodriguez-Yam, G., Davis, R. A., and Scharf, L. L. (2004). “Efficient Gibbs sampling of truncated multivariate normal with application to constrained linear regression.”
*Technical report*. - Rue, H. (2001). “Fast sampling of Gaussian Markov random fields.”
*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*, 63(2): 325–338. - Schmidt, M. (2009). “Linearly constrained Bayesian matrix factorization for blind source separation.” In
*NIPS*, 1624–1632. - Tong, Y. L. (2012).
*The Multivariate Normal Distribution*. Springer Science & Business Media. - Train, K. E. (2009).
*Discrete Choice Methods With Simulation*. Cambridge University Press. - Zhou, M., Cong, Y., and Chen, B. (2016). “Augmentable gamma belief networks.”
*Journal of Machine Learning Research*, 17(163): 1–44.Mathematical Reviews (MathSciNet): MR3555054 - Zhou, M., Hannah, L., Dunson, D. B., and Carin, L. (2012). “Beta-negative binomial process and Poisson factor analysis.” In
*AISTATS*, 1462–1471.

#### Supplemental materials

- Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions: Supplementary Material. Digital Object Identifier: doi:10.1214/17-BA1052SUPP

### More like this

- Tracy–Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices

El Karoui, Noureddine, The Annals of Probability, 2007 - A scatter matrix estimate based on the zonotope

Koshevoy, Gleb A., Möttönen, Jyrki, and Oja, Hannu, The Annals of Statistics, 2003 - Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials

Fox, Colin and Parker, Albert, Bernoulli, 2017

- Tracy–Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices

El Karoui, Noureddine, The Annals of Probability, 2007 - A scatter matrix estimate based on the zonotope

Koshevoy, Gleb A., Möttönen, Jyrki, and Oja, Hannu, The Annals of Statistics, 2003 - Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials

Fox, Colin and Parker, Albert, Bernoulli, 2017 - ROP: Matrix recovery via rank-one projections

Cai, T. Tony and Zhang, Anru, The Annals of Statistics, 2015 - The General Projected Normal Distribution of Arbitrary Dimension: Modeling and Bayesian Inference

Hernandez-Stumpfhauser, Daniel, Breidt, F. Jay, and van der Woerd, Mark J., Bayesian Analysis, 2017 - Slicing Regression: A Link-Free Regression Method

Duan, Naihua and Li, Ker-Chau, The Annals of Statistics, 1991 - Covariance Estimation: The GLM and Regularization Perspectives

Pourahmadi, Mohsen, Statistical Science, 2011 - A Distribution-Free Approach to Stochastic Efficiency Measurement with Inclusion of Expert Knowledge

Khoo-Fazari, Kerry, Yang, Zijiang, and Paradi, Joseph C., Journal of Applied Mathematics, 2013 - On the behavior of the covariance matrices in a multivariate central limit theorem under some mixing conditions

Bradley, Richard C., Illinois Journal of Mathematics, 2012 - A two-dimensional, two-sided Euler inversion algorithm with computable error bounds and its financial applications

Cai, Ning and Shi, Chao, Stochastic Systems, 2014