The Annals of Statistics

A Bayesian approach for envelope models

Kshitij Khare, Subhadip Pal, and Zhihua Su

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

The envelope model is a new paradigm to address estimation and prediction in multivariate analysis. Using sufficient dimension reduction techniques, it has the potential to achieve substantial efficiency gains compared to standard models. This model was first introduced by [Statist. Sinica 20 (2010) 927–960] for multivariate linear regression, and has since been adapted to many other contexts. However, a Bayesian approach for analyzing envelope models has not yet been investigated in the literature. In this paper, we develop a comprehensive Bayesian framework for estimation and model selection in envelope models in the context of multivariate linear regression. Our framework has the following attractive features. First, we use the matrix Bingham distribution to construct a prior on the orthogonal basis matrix of the envelope subspace. This prior respects the manifold structure of the envelope model, and can directly incorporate prior information about the envelope subspace through the specification of hyperparamaters. This feature has potential applications in the broader Bayesian sufficient dimension reduction area. Second, sampling from the resulting posterior distribution can be achieved by using a block Gibbs sampler with standard associated conditionals. This in turn facilitates computationally efficient estimation and model selection. Third, unlike the current frequentist approach, our approach can accommodate situations where the sample size is smaller than the number of responses. Lastly, the Bayesian approach inherently offers comprehensive uncertainty characterization through the posterior distribution. We illustrate the utility of our approach on simulated and real datasets.

Article information

Source
Ann. Statist., Volume 45, Number 1 (2017), 196-222.

Dates
Received: July 2014
Revised: January 2016
First available in Project Euclid: 21 February 2017

Permanent link to this document
https://projecteuclid.org/euclid.aos/1487667621

Digital Object Identifier
doi:10.1214/16-AOS1449

Mathematical Reviews number (MathSciNet)
MR3611490

Zentralblatt MATH identifier
1367.62174

Subjects
Primary: 62F15: Bayesian inference 62F30: Inference under constraints
Secondary: 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40] 62H12: Estimation

Keywords
Sufficient dimension reduction envelope model Gibbs sampling Stiefel manifold matrix Bingham distribution

Citation

Khare, Kshitij; Pal, Subhadip; Su, Zhihua. A Bayesian approach for envelope models. Ann. Statist. 45 (2017), no. 1, 196--222. doi:10.1214/16-AOS1449. https://projecteuclid.org/euclid.aos/1487667621


Export citation

References

  • [1] Asmussen, S. and Glynn, P. W. (2011). A new proof of convergence of MCMC via the ergodic theorem. Statist. Probab. Lett. 81 1482–1485.
  • [2] Bingham, C. (1974). An antipodally symmetric distribution on the sphere. Ann. Statist. 2 1201–1225.
  • [3] Conway, J. B. (1990). A Course in Functional Analysis, 2nd ed. Graduate Texts in Mathematics 96. Springer, New York.
  • [4] Cook, R. D., Helland, I. S. and Su, Z. (2013). Envelopes and partial least squares regression. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 851–877.
  • [5] Cook, R. D., Li, B. and Chiaromonte, F. (2010). Envelope models for parsimonious and efficient multivariate linear regression. Statist. Sinica 20 927–960.
  • [6] Cook, R. D. and Su, Z. (2013). Scaled envelopes: Scale-invariant and efficient estimation in multivariate linear regression. Biometrika 100 939–954.
  • [7] Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL.
  • [8] Hoff, P. D. (2009). Simulation of the matrix Bingham–von Mises–Fisher distribution, with applications to multivariate and relational data. J. Comput. Graph. Statist. 18 438–456.
  • [9] Khare, K., Pal, S. and Su, Z. (2016). Supplement to “A Bayesian approach for envelope models.” DOI:10.1214/16-AOS1449SUPP.
  • [10] Mao, K., Liang, F. and Mukherjee, S. (2010). Supervised dimension reduction using Bayesian mixture modeling. In International Conference on Artificial Intelligence and Statistics 501–508.
  • [11] Reich, B. J., Bondell, H. D. and Li, L. (2011). Sufficient dimension reduction via Bayesian mixture modeling. Biometrics 67 886–895.
  • [12] Rossi, P. E., Allenby, G. M. and McCulloch, R. (2005). Bayesian Statistics and Marketing. Wiley, Chichester.
  • [13] Sameh, A. H. and Wisniewski, J. A. (1982). A trace minimization algorithm for the generalized eigenvalue problem. SIAM J. Numer. Anal. 19 1243–1259.
  • [14] Schott, J. R. (2013). On the likelihood ratio test for envelope models in multivariate linear regression. Biometrika 100 531–537.
  • [15] Su, Z. and Cook, D. (2012). Inner envelopes: Efficient estimation in multivariate linear regression. Biometrika 99 687–702.
  • [16] Su, Z. and Cook, R. D. (2011). Partial envelopes for efficient estimation in multivariate linear regression. Biometrika 98 133–146.
  • [17] Su, Z. and Cook, R. D. (2013). Estimation of multivariate means with heteroscedastic errors using envelope models. Statist. Sinica 23 213–230.
  • [18] Tokdar, S. T., Zhu, Y. M. and Ghosh, J. K. (2010). Bayesian density regression with logistic Gaussian process and subspace projection. Bayesian Anal. 5 319–344.

Supplemental materials

  • Supplement to “A Bayesian approach for envelope models”. The supplement [9] provides additional details and proofs for many of the results in the authors’ paper.