Bayesian Analysis

Bayesian density regression with logistic Gaussian process and subspace projection

Jayanta K. Ghosh, Surya T. Tokdar, and Yu M. Zhu

Full-text: Access has been disabled (more information)

Abstract

We develop a novel Bayesian density regression model based on logistic Gaussian processes and subspace projection. Logistic Gaussian processes provide an attractive alternative to the popular stick-breaking processes for modeling a family of conditional densities that vary smoothly in the conditioning variable. Subspace projection offers dimension reduction of predictors through multiple linear combinations, offering an alternative to the zeroing out theme of variable selection. We illustrate that logistic Gaussian processes and subspace projection combine well to produce a computationally tractable and theoretically sound density regression procedure that offers good out of sample prediction, accurate estimation of subspace projection and satisfactory estimation of subspace dimensionality. We also demonstrate that subspace projection may lead to better prediction than variable selection when predictors are well chosen and possibly dependent on each other, each having a moderate influence on the response.

Article information

Source
Bayesian Anal. Volume 5, Number 2 (2010), 319-344.

Dates
First available in Project Euclid: 20 June 2012

Permanent link to this document
http://projecteuclid.org/euclid.ba/1340218341

Digital Object Identifier
doi:10.1214/10-BA605

Mathematical Reviews number (MathSciNet)
MR2719655

Zentralblatt MATH identifier
1330.62182

Keywords
Bayesian Inference Semiparametric Model Posterior Consistency Gaussian Process Markov Chain Monte Carlo Dimension Reduction

Citation

Tokdar, Surya T.; Zhu, Yu M.; Ghosh, Jayanta K. Bayesian density regression with logistic Gaussian process and subspace projection. Bayesian Anal. 5 (2010), no. 2, 319--344. doi:10.1214/10-BA605. http://projecteuclid.org/euclid.ba/1340218341.


Export citation

References

  • Adams, R. P., Murray, I., and MacKay, D. J. C. (2009). "The Gaussian process density sampler." In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L. (eds.), Advances in Neural Information Processing Systems 21 (NIPS 2008), 9–16.
  • Banerjee, S., Gelfand, A. E., Finley, A. O., and Sang, H. (2008). "Gaussian predictive process models for large spatial data sets." Journal of the Royal Statistical Society - Series B, 70: 825–848.
    Mathematical Reviews (MathSciNet): MR2523906
    Zentralblatt MATH: 05563371
    Digital Object Identifier: doi:10.1111/j.1467-9868.2008.00663.x
  • Chen, C. H. and Li, K. C. (1998). "Can SIR be as Popular as Multiple Linear Regression?" Statistica Sinica, 8: 289–316.
    Mathematical Reviews (MathSciNet): MR1624402
    Zentralblatt MATH: 0897.62069
  • Chib, S. and Jeliazkov, I. (2001). "Marginal Likelihood from the Metropolis-Hastings Output." Journal of the American Statistical Association, 96: 270–281.
    Mathematical Reviews (MathSciNet): MR1952737
    Zentralblatt MATH: 1015.62020
    Digital Object Identifier: doi:10.1198/016214501750332848
  • Chung, Y. and Dunson, D. B. (2009). "Nonparametric Bayes conditional distribution modeling with variable selection." Journal of the American Statistical Association, 104(488): 1646–1660.
    Mathematical Reviews (MathSciNet): MR2750582
    Zentralblatt MATH: 1205.62039
    Digital Object Identifier: doi:10.1198/jasa.2009.tm08302
  • Cook, R. D. (1994). "Using dimension-reduction subspaces to identify important inputs in models of physical systems." In Proceedings of the Section on Physical Engineering Sciences, 18–25. Washington: American Statistical Association.
  • Cook, R. D. and Weisberg, S. (1991). "Discussion of Sliced Inverse Regression for Dimension Reduction by K. C. Li." Journal of the American Statistical Association, 86: 328–332.
    Mathematical Reviews (MathSciNet): MR1137117
    Zentralblatt MATH: 0742.62044
  • Dunson, D. B. and Park, J. H. (2008). "Kernel Stick-breaking Processes." Biometrika, 95: 307–323.
    Mathematical Reviews (MathSciNet): MR2521586
    Zentralblatt MATH: 05563397
    Digital Object Identifier: doi:10.1093/biomet/asn012
  • Dunson, D. B., Pillai, N., and Park, J. H. (2007). "Bayesian Density Regression." Journal of the Royal Statistical Society - Series B, 69: 163–183.
    Mathematical Reviews (MathSciNet): MR2325270
    Zentralblatt MATH: 1120.62025
    Digital Object Identifier: doi:10.1111/j.1467-9868.2007.00582.x
  • Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004). Bayesian Data Analysis. Chapman & Hall/CRC, 2 edition.
    Mathematical Reviews (MathSciNet): MR2027492
  • Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. V. (1999). "Posterior consistency of Dirichlet mixtures in density estimation." The Annals of Statistics, 27: 143–158.
    Mathematical Reviews (MathSciNet): MR1701105
    Zentralblatt MATH: 0932.62043
    Digital Object Identifier: doi:10.1214/aos/1018031105
    Project Euclid: euclid.aos/1018031105
  • Ghosal, S. and Roy, A. (2006). "Posterior Consistency of Gaussian Process Prior for Nonparametric Binary Regression." The Annals of Statistics, 34: 2413–2429.
    Mathematical Reviews (MathSciNet): MR2291505
    Zentralblatt MATH: 1106.62039
    Digital Object Identifier: doi:10.1214/009053606000000795
    Project Euclid: euclid.aos/1169571802
  • Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer-Verlag.
    Mathematical Reviews (MathSciNet): MR1992245
    Zentralblatt MATH: 1029.62004
  • Griffin, J. E. and Steel, M. F. J. (2006). "Order-Based Dependent Dirichlet Processes." Journal of the American Statistical Association, 101: 179–194.
    Mathematical Reviews (MathSciNet): MR2268037
    Zentralblatt MATH: 1118.62360
    Digital Object Identifier: doi:10.1198/016214505000000727
  • Harrison, D. and Rubinfeld, D. L. (1978). "Hedonic Housing Prices and the Demand for Clean Air." Journal of Environmental Economics and Management, 5: 81–102.
    Zentralblatt MATH: 0375.90023
  • Lenk, P. J. (1988). "The Logistic Normal Distribution for Bayesian, Nonparametric, Predictive Densities." Journal of the American Statistical Association, 83: 509–516.
    Mathematical Reviews (MathSciNet): MR971380
    Zentralblatt MATH: 0648.62034
  • –- (1991). "Towards a Practicable Bayesian Nonparametric Density Estimator." Biometrika, 78: 531–543.
    Mathematical Reviews (MathSciNet): MR1130921
    Zentralblatt MATH: 0737.62035
    Digital Object Identifier: doi:10.1093/biomet/78.3.531
  • –- (2003). "Bayesian Semiparametric Density Estimation and Model Verification Using a Logistic Gaussian Process." Journal of Computational and Graphical Statistics, 12: 548–565.
    Mathematical Reviews (MathSciNet): MR2002635
    Digital Object Identifier: doi:10.1198/1061860032021
  • Li, K. C. (1991). "Sliced Inverse Regression for Dimension Reduction (with discussions)." Journal of the American Statistical Association, 86: 316–342.
    Mathematical Reviews (MathSciNet): MR1137117
    Zentralblatt MATH: 0742.62044
  • MacEachern, S. M. (1999). "Dependent Nonparametric Processes." In Proceedings of the Section on Bayesian Statistical Science, 50–55. Alexandria, VA: American Statistical Association.
  • –- (2000). "Dependent Dirichlet Processes." Ohio State University, Dept. of Statistics Technical Report.
  • Mattila, P. (1995). Geometry of Sets and Measures in Euclidean Spaces. New York: Cambridge University Press.
    Mathematical Reviews (MathSciNet): MR1333890
    Zentralblatt MATH: 0819.28004
  • Neal, R. M. (1996). Bayesian Learning for Neural Networks. New York: Springer Verlag.
    Zentralblatt MATH: 0888.62021
  • Tokdar, S. T. (2007). "Towards a Faster Implementation of Density Estimation with Logistic Gaussian Process Priors." Journal of Computational and Graphical Statistics, 16: 633–655.
    Mathematical Reviews (MathSciNet): MR2351083
    Digital Object Identifier: doi:10.1198/106186007X210206
  • Tokdar, S. T. and Ghosh, J. K. (2007). "Posterior consistency of logistic Gaussian process priors in density estimation." Journal of Statistical Planning and Inference, 137: 34–42.
    Mathematical Reviews (MathSciNet): MR2292838
    Zentralblatt MATH: 1098.62041
    Digital Object Identifier: doi:10.1016/j.jspi.2005.09.005
  • van der Vaart, A. W. and van Zanten, H. (2008). "Rates of contraction of posterior distributions based on Gaussian process priors." The Annals of Statistics, 36: 1435–1463.
    Mathematical Reviews (MathSciNet): MR2418663
    Zentralblatt MATH: 1141.60018
    Digital Object Identifier: doi:10.1214/009053607000000613
    Project Euclid: euclid.aos/1211819570
  • Xia, Y. (2007). "A constructive approach to the estimation of dimension reduction directions." The Annals of Statistics, 35: 2654–2690.
    Mathematical Reviews (MathSciNet): MR2382662
    Zentralblatt MATH: 05241119
    Digital Object Identifier: doi:10.1214/009053607000000352
    Project Euclid: euclid.aos/1201012976
  • Zeng, P. (2008). "Determining the dimension of the central subspace and central mean subspace." Biometrika, 95: 469–479.
    Mathematical Reviews (MathSciNet): MR2521593
    Zentralblatt MATH: 05563407
    Digital Object Identifier: doi:10.1093/biomet/asn002
  • Zhu, Y. and Zeng, P. (2006). "Fourier Methods for Estimating the Central Subspace and the Central Mean Subspace in Regression." Journal of the American Statistical Association, 101: 1638–1651.
    Mathematical Reviews (MathSciNet): MR2279485
    Zentralblatt MATH: 1171.62325
    Digital Object Identifier: doi:10.1198/016214506000000140

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more