Electronic Journal of Statistics

Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks

Tristan Gray-Davies, Chris C. Holmes, and François Caron

Full-text: Open access

Abstract

We present a novel Bayesian nonparametric regression model for covariates $X$ and continuous response variable $Y\in\mathbb{R}$. The model is parametrized in terms of marginal distributions for $Y$ and $X$ and a regression function which tunes the stochastic ordering of the conditional distributions $F(y|x)$. By adopting an approximate composite likelihood approach, we show that the resulting posterior inference can be decoupled for the separate components of the model. This procedure can scale to very large datasets and allows for the use of standard, existing, software from Bayesian nonparametric density estimation and Plackett-Luce ranking estimation to be applied. As an illustration, we show an application of our approach to a US Census dataset, with over 1,300,000 data points and more than 100 covariates.

Article information

Source
Electron. J. Statist., Volume 10, Number 2 (2016), 1807-1828.

Dates
Received: December 2014
First available in Project Euclid: 18 July 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1468849964

Digital Object Identifier
doi:10.1214/15-EJS1032

Mathematical Reviews number (MathSciNet)
MR3522661

Zentralblatt MATH identifier
06624502

Keywords
Bayesian nonparametrics composite likelihood Plackett-Luce Pólya Tree Dirichlet process mixtures

Citation

Gray-Davies, Tristan; Holmes, Chris C.; Caron, François. Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks. Electron. J. Statist. 10 (2016), no. 2, 1807--1828. doi:10.1214/15-EJS1032. https://projecteuclid.org/euclid.ejs/1468849964


Export citation

References

  • Caron, F., Davy, M., & Doucet, A., 2007. Generalized Polya urn for time-varying Dirichlet process mixtures. In:, 23rd Conference on Uncertainty in Artificial Intelligence (UAI’2007).
  • Caron, F., Davy, M., Doucet, A., Duflos, E., & Vanheeghe, P., 2008. Bayesian inference for linear dynamic models with Dirichlet process mixtures., IEEE Transactions on Signal Processing, 56(1), 71–84.
  • Chipman, H.A., George, E.I., & McCulloch, R.E., 2010. BART: Bayesian additive regression trees., Ann. Appl. Stat., 4(1), 266–298.
  • Chung, Y. & Dunson, D.B., 2009. Nonparametric Bayes conditional distribution modeling with variable selection., Journal of the American Statistical Association, 104(488).
  • Cox, D.R., 1972. Regression models and life-tables., Journal of the Royal Statistical Society. Series B (Methodological), 34(2), 187–220.
  • Cox, D.R. & Reid, N., 2004. A note on pseudolikelihood constructed from marginal densities., Biometrika, 91(3), 729–737.
  • Hoff, P.D., 2007. Extending the rank likelihood for semiparametric copula estimation., Ann. Appl. Stat., 1(1), 265–283.
  • De Iorio, M., Müller, P., Rosner, G.L., & MacEachern, S.N., 2004. An ANOVA model for dependent random measures., Journal of the American Statistical Association, 99(465), 205–215.
  • Denison, D.G.T., Holmes, C.C., Mallick, B.K., & Smith, A.F.M., 2002., Bayesian Methods for Nonlinear Classification and Regression. John Wiley & Sons.
  • Dunson, D.B. & Park, J.-H., 2008. Kernel stick-breaking processes., Biometrika, 95(2), 307–323.
  • Dunson, D.B., Pillai, N., & Park, J.-H., 2007. Bayesian density regression., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2), 163–183.
  • Escobar, M.D. & West, M., 1995. Bayesian density estimation and inference using mixtures., Journal of the American Statistical Association, 90(430), 577–588.
  • Ferguson, T.S., 1973. A Bayesian analysis of some nonparametric problems., The Annals of Statistics, 1(2), 209–230.
  • Ferguson, T.S., 1974. Prior distributions on spaces of probability measures., The Annals of Statistics, 2(4), 615–629.
  • Gelfand, A. & Kottas, A., 2003. Bayesian semiparametric regression for median residual life., Scandinavian Journal of Statistics, 30(4), 651–665.
  • Ghosal, S. & Van der Vaart, A.W., 2013., Fundamentals of Nonparametric Bayesian Inference. Cambridge University Press, New York.
  • Griffin, J.E. & Steel, M.F.J., 2006. Order-based dependent Dirichlet processes., Journal of the American Statistical Association, 101(473), 179–194.
  • Hannah, L.A., Blei, D., & Powell, W.B., 2011. Dirichlet process mixtures of generalized linear models., The Journal of Machine Learning Research, 12, 1923–1953.
  • Hjort, N.L., Holmes, C.C., Müller, P., & Walker, S.G., 2010., Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.
  • Jara, A., 2007. Applied Bayesian non- and semi-parametric inference using DPpackage., R News, 7(3), 17–26.
  • Jara, A., Hanson, T., Quintana, F., Müller, P., & Rosner, G., 2011. DPpackage: Bayesian semi- and nonparametric modeling in R., Journal of Statistical Software, 40(5), 1–30.
  • Kim, Y., 2006. The Bernstein–von Mises theorem for the proportional hazard model., The Annals of Statistics, 34(4), 1678–1700.
  • Kottas, A. & Gelfand, A.E., 2001. Bayesian semiparametric median regression modeling., Journal of the American Statistical Association, 96(456), 1458–1468.
  • Lavine, M., 1992. Some aspects of Polya tree distributions for statistical modelling., The Annals of Statistics, 20(3), 1222–1235.
  • Lavine, M., 1994. More aspects of Polya tree distributions for statistical modelling., The Annals of Statistics, 22(3), 1161–1176.
  • Lavine, M. & Mockus, A., 1995. A nonparametric Bayes method for isotonic regression., Journal of Statistical Planning and Inference, 46(2), 235– 248.
  • Lindsay, B.G., 1988. Composite likelihood methods., Contemporary Mathematics, 80(1), 221–39.
  • Lo, A.Y., 1984. On a class of Bayesian nonparametric estimates: I. Density estimates., The Annals of Statistics, 12(1), 351–357.
  • Luce, R.D., 1959., Individual Choice Behavior: A Theoretical Analysis. John Wiley and Sons.
  • MacEachern, S.N., 1999. Dependent nonparametric processes. In:, Proceedings of the Bayesian Statistical Science Section. American Statistical Association.
  • MacEachern, S.N. & Müller, P., 1998. Estimating mixture of Dirichlet process models., Journal of Computational and Graphical Statistics, 7(2), 223–238.
  • Mauldin, R.D., Sudderth, W.D., & Williams, S.C., 1992. Polya trees and random distributions., The Annals of Statistics, 20(3), 1203–1221.
  • Müller, P. & Quintana, F.A., 2004. Nonparametric Bayesian data analysis., Statistical Science, 19(1), 95–110.
  • Müller, P., Erkanli, A., & West, M., 1996. Bayesian curve fitting using multivariate normal mixtures., Biometrika, 83(1), 67–79.
  • Murray, J.S., Dunson, D.B., Carin, L., & Lucas, J.E., 2013. Bayesian Gaussian copula factor models for mixed data., Journal of the American Statistical Association, 108(502), 656–665.
  • Neal, R.M., 2000. Markov chain sampling methods for Dirichlet process mixture models., Journal of Computational and Graphical Statistics, 9(2), 249–265.
  • Pati, D. & Dunson, D.B., 2014. Bayesian nonparametric regression with varying residual density., Annals of the Institute of Statistical Mathematics, 66(1), 1–31.
  • Pauli, F., Racugno, W., & Ventura, L., 2011. Bayesian composite marginal likelihoods., Statistica Sinica, 21(1), 149.
  • Plackett, R.L., 1975. The analysis of permutations., Journal of the Royal Statistical Society. Series C (Applied Statistics), 24(2), 193–202.
  • Rasmussen, C.E., 2006., Gaussian Processes for Machine Learning. MIT Press.
  • Ribatet, M., Cooley, D., & Davison, A.C., 2012. Bayesian inference from composite likelihoods, with an application to spatial extremes., Statistica Sinica, 22, 813–845.
  • Shahbaba, B. & Neal, R., 2009. Nonlinear models using Dirichlet process mixtures., Journal of Machine Learning Research, 10, 1829–1850.
  • Trippa, L., Müller, P., & Johnson, W., 2011. The multivariate beta process and an extension of the Polya tree model., Biometrika, 98(1), 17–34.
  • Varin, C., Reid, N., & Firth, D., 2011. An overview of composite likelihood methods., Statistica Sinica, 21(1), 5–42.
  • Wade, S., Dunson, D.B., Petrone, S., & Trippa, L., 2014. Improving prediction from Dirichlet process mixtures via enrichment., Journal of Machine Learning Research, 15, 1041–1071.