Electronic Journal of Statistics

Posterior asymptotics in the supremum $L_{1}$ norm for conditional density estimation

Pierpaolo De Blasi and Stephen G. Walker

Full-text: Open access

Abstract

In this paper we study posterior asymptotics for conditional density estimation in the supremum $L_{1}$ norm. Compared to the expected $L_{1}$ norm, the supremum $L_{1}$ norm allows accurate prediction at any designated conditional density. We model the conditional density as a regression tree by defining a data dependent sequence of increasingly finer partitions of the predictor space and by specifying the conditional density to be the same across all predictor values in a partition set. Each conditional density is modeled independently so that the prior specifies a type of dependence between conditional densities which disappears after a certain number of observations have been observed. The rate at which the number of partition sets increases with the sample size determines when the dependence between pairs of conditional densities is set to zero and, ultimately, drives posterior convergence at the true data distribution.

Article information

Source
Electron. J. Statist., Volume 10, Number 2 (2016), 3219-3246.

Dates
Received: December 2015
First available in Project Euclid: 16 November 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1479287219

Digital Object Identifier
doi:10.1214/16-EJS1191

Mathematical Reviews number (MathSciNet)
MR3572847

Zentralblatt MATH identifier
1357.62199

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G08: Nonparametric regression

Keywords
Nonparametric Bayesian inference posterior asymptotics conditional density estimation regression tree model

Citation

De Blasi, Pierpaolo; Walker, Stephen G. Posterior asymptotics in the supremum $L_{1}$ norm for conditional density estimation. Electron. J. Statist. 10 (2016), no. 2, 3219--3246. doi:10.1214/16-EJS1191. https://projecteuclid.org/euclid.ejs/1479287219


Export citation

References

  • Castillo, I. (2014). On Bayesian supremum norm contraction rates., Ann. Statist., 42(5):2058–2091.
  • Chipman, H. A., George, E. I., and McCulloch, R. E. (1998). Bayesian cart model search., J. Amer. Statist. Assoc., 93:935–948.
  • De Blasi, P., James, L. F., and Lau, J. W. (2010). Bayesian nonparametric estimation and consistency of mixed multinomial logit choice models., Bernoulli, 16:679–704.
  • Efromovich, S. (2007). Conditional density estimation in a regression setting., The Annals of Statistics, 35(6):2504–2535.
  • Ghosal, S., Ghosh, J. K., and van der Vaart, A. W. (2000). Convergence rates of posterior distributions., Ann. Statist., 28(2):500–531.
  • Ghosal, S. and Roy, A. (2006). Posterior consistency of Gaussian process prior for nonparametric binary regression., Ann. Statist., 34:2413–2429.
  • Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities., Ann. Statist., 29(5):1233–1263.
  • Ghosal, S. and van der Vaart, A. W. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities., Ann. Statist., 35(5):697–723.
  • Ginè, E. and Nickl, R. (2011). Rates of contraction for posterior distributions in $l^r$-metrics, $1\leq r\leq\infty$., Ann. Statist., 39:2883–2911.
  • Hoffman, M., Rousseau, J., and Schmidt-Hieber, J. (2015). On adaptive posterior contraction rates., Ann. Statist., 43(5):2259–2295.
  • Kruijer, W., Rousseau, J., and van der Vaart, A. W. (2010). Adaptive Bayesian density estimation with location-scale mixtures., Electronic Journal of Statistics, 4:1225–1257.
  • Lijoi, A., Prünster, I., and Walker, S. G. (2005). On consistency of nonparametric normal mixtures for bayesian density estimation., J. Amer. Statist. Assoc., 100(472):1292–1296.
  • Ma, L. (2012). Recursive partitioning and bayesian inference on conditional distributions. Technical report, Duke Statistical Science Discussion Paper, 2012-03.
  • Norets, A. and Pati, D. (2014). Adaptive Bayesian estimation of conditional densities. Technical report, arXiv:1408.5355v2.
  • Pati, D., Dunson, D. B., and Tokdar, S. T. (2013). Posterior consistency in conditional distribution estimation., J. Multivariate Anal., 116:456–472.
  • Shen, W. and Ghosal, S. (2016). Adaptive bayesian density regression for high-dimensional data., Bernoulli, 22(1):396–420.
  • Shen, W., Tokdar, S. T., and Ghosal, S. (2013). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures., Biometrika, 100:623–640.
  • Shively, T. S., Sager, T. W., and Walker, S. G. (2009). A Bayesian approach to non-parametric monotone function estimation., J. R. Stat. Soc. Ser. B, 71:159–175.
  • Tang, Y. and Ghosal, S. (2007). Posterior consistency of dirichlet mixture for estimating a transition density., J. Statist. Plann. Inference, 137:1711–1726.
  • Tokdar, S. T., Zhu, Y., and Ghosh, J. K. (2010). Bayesian density regression with logistic Gaussian process and subspace projection., Bayesian Analysis, 5:1–26.
  • Walker, S. G. (2004). New approaches to Bayesian consistency., Ann. Statist., 32:2028–2043.
  • Walker, S. G., Lijoi, A., and Prünster, I. (2007). On rates of convergence for posterior distributions in infinite-dimensional models., Ann. Statist., 35:738–746.
  • Xiang, F. and Walker, S. G. (2013). Bayesian consistency for regression models under a supremum distance., J. Statist. Plann. Inference, 143:468–478.
  • Yoo, W. W. and Ghosal, S. (2016). Supremum norm posterior contraction and credible sets for nonparametric multivariate regression., Ann. Statist., 44(3):1069–1102.