Bayesian Analysis

Nonparametric Bayesian models through probit stick-breaking processes

David B. Dunson and Abel Rodríguez

Full-text: Open access

Abstract

We describe a novel class of Bayesian nonparametric priors based on stick-breaking constructions where the weights of the process are constructed as probit transformations of normal random variables. We show that these priors are extremely flexible, allowing us to generate a great variety of models while preserving computational simplicity. Particular emphasis is placed on the construction of rich temporal and spatial processes, which are applied to two problems in finance and ecology.

Article information

Source
Bayesian Anal. Volume 6, Number 1 (2011), 145-177.

Dates
First available in Project Euclid: 13 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1339611944

Digital Object Identifier
doi:10.1214/11-BA605

Mathematical Reviews number (MathSciNet)
MR2781811

Zentralblatt MATH identifier
1330.62120

Subjects
Primary: 62F15: Bayesian inference
Secondary: 60G57: Random measures 62G99: None of the above, but in this section 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 62M30: Spatial processes 62P12: Applications to environmental and related topics

Keywords
Nonparametric Bayes Random Probability Measure Stick-breaking Prior Mixture Model Data Augmentation Spatial Data Time Series

Citation

Rodríguez, Abel; Dunson, David B. Nonparametric Bayesian models through probit stick-breaking processes. Bayesian Anal. 6 (2011), no. 1, 145--177. doi:10.1214/11-BA605. https://projecteuclid.org/euclid.ba/1339611944


Export citation

References

  • Agresti, A. (1990). Categorical Data Analysis. New York: Wiley.
  • Albert, J. H. and Chib, S. (2001). "Sequential ordinal modeling with applications to survival data." Biometrics, 57: 829–836.
  • Banerjee, S. and Gelfand, A. E. (2003). "On smoothness properties of spatial processes." Journal of Multivariate Analysis, 84: 85–100.
  • Blackwell, D. and MacQueen, J. B. (1973). "Ferguson Distribution via Pó"lya Urn Schemes. The Annals of Statistics, 1: 353–355.
  • Caron, F., Davy, M., and Doucet, A. (2007). "Generalized Pó"lya urn for time-varying Dirichlet process mixtures. In Proceedings of the Twenty-Third Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-07), 33–40. Corvallis, Oregon: AUAI Press.
  • Caron, F., Davy, M., Doucet, A., Duflos, E., and Vanheeghe, P. (2008). "Bayesian inference for linear dynamic models with Dirichlet process mixtures." IEEE Transactions on Signal Processing, 56: 71–84.
  • Carter, C. K. and Kohn, R. (1994). "On Gibbs sampling for state space models." Biometrika, 81: 541–553.
  • Chib, S. and Hamilton, B. H. (2002). "Semiparametric Bayes Analysis of Longitudinal Data Treatment Models." Journal of Econometrics, 110: 67–89.
  • Chung, Y. and Dunson, D. B. (2009). "Nonparametric Bayes conditional distribution modeling with variable selection." Journal of American Statistical Association, 104: 1646–1660.
  • DeIorio, M., Müller, P., Rosner, G. L., and MacEachern, S. N. (2004). "An ANOVA" Model for Dependent Random Measures. Journal of the American Statistical Association, 99: 205–215.
  • Duan, J. A., Guindani, M., and Gelfand, A. E. (2007). "Generalized Spatial Dirichlet Process Models." Biometrika, 94: 809–825.
  • Dunson, D. (2006). "Bayesian dynamic modeling of latent trait distributions." Biostatistics, 7: 551–568.
  • Dunson, D. B., Herring, A. H., and Mulheri-Engel, S. A. (2008). "Bayesian selection and clustering of polymorphisms in functionally-related genes." Journal of the Royal Statistical Society, 103: 534–546.
  • Dunson, D. B., Pillai, N., and Park, J.-H. (2007). "Bayesian density regression." Journal of the Royal Statistical Society, Series B., 69: 163–183.
  • Escobar, M. D. (1994). "Estimating Normal Means with a Dirichlet Process Prior." Journal of the American Statistical Association, 89: 268–277.
  • Escobar, M. D. and West, M. (1995). "Bayesian Density Estimation and Inference Using Mixtures." Journal of American Statistical Association, 90: 577–588.
  • Ferguson, T. S. (1973). "A Bayesian Analysis of Some Nonparametric Problems." Annals of Statistics, 1: 209–230.
  • –- (1974). "Prior Distributions on Spaces of Probability Measures." Annals of Statistics, 2 : 615–629.
  • Figueiredo, M. A., Cheng, D. S., and Murino, V. (2007). "Clustering Under Prior Knowledge with Application to Image Segmentation." In Schölkopf, B., Platt, J., and Hoffman, T. (eds.), Advances in Neural Information Processing Systems 19, 401–408. Cambridge, MA: MIT Press.
  • Figueiredo, M. A. T. (2005). "Bayesian image segmentation using Gaussian field priors." In Rangarajan, A., Vemuri, B., and Yuille, A. L. (eds.), Energy Minimization Methods in Computer Vision and Pattern Recognition, 74–89. Springer - Verlag, Berlin Heidelberg.
  • Frühwirth-Schnatter, S. (1994). "Data augmentation and dynamic linear models." Journal of Time Series Analysis, 15: 183–202.
  • Gelfand, A. E. and Kottas, A. (2002). "A Computational Approach for Full Nonparameteric Bayesian Inference under Dirichlet Process Mixture Modles." Journal of Computational and Graphical Statistics, 11: 289–305.
  • Gelfand, A. E., Kottas, A., and MacEachern, S. N. (2005). "Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing." Journal of the American Statistical Association, 100: 1021–1035.
  • Gelman, A. and Rubin, D. (1992). "Inferences from iterative simulation using multiple sequences." Statistical Science, 7: 457–472.
  • Griffin, J. (2008). "The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference." Technical report, Institute of Mathematics, Statistics and Actuarial Science, University of Kent.
  • Griffin, J. E. and Steel, M. F. J. (2006). "Order-Based Dependent Dirichlet Processes." Journal of the American Statistical Association, 101: 179–194.
  • –- (2010). "Stick-breaking autoregressive processes." Journal of Econometrics, (Forthcoming).
  • Han, D. F., Li, W. H., and Li, Z. C. (2008). "Semantic image classification using statistical local spatial relations model." Multimedia Tools and Applications, 39: 169–188.
  • Hirano, K. (2002). "Semiparametric Bayesian Inference in Autoregressive Panel Data Models." Econometrica, 70: 781–799.
  • Hjort, N. (2000). "Bayesian analysis for a generalized Dirichlet process prior." Technical report, University of Oslo.
  • Ishwaran, H. and James, L. F. (2001). "Gibbs Sampling Methods for Stick-Breaking Priors." Journal of the American Statistical Association, 96: 161–173.
  • –- (2003). "Some further developments for stick-breaking priors: finite and infinite clustering and classification." Sankya, 65: 577–592.
  • Ishwaran, H. and Zarepour, M. (2002). "Dirichlet Prior Sieves in Finite Normal Mixtures." Statistica Sinica, 12: 941–963.
  • Jacquier, E., Polson, N. G., and Rossi, P. E. (1994). "Bayesian Analysis of Stochastic Volatility Models." Journal of business and Economic Statistics, 12: 371–389.
  • Kacperczyk, M., Damien, P., and Walker, S. G. (2003). "A new class of Bayesian Semiparametric Models with Applications to Option Pricing." Technical report, University of Michigan Bussiness School.
  • Kaufman, K. (1996). Lives of North American Birds. Houghton Mifflin.
  • Kent, J. T. (1989). "Continuity properties for random fields." Annals of Probability, 17: 1432–1440.
  • Kim, S., Shephard, N., and Chib, S. (1998). "Stochastic Volatility: Likelihood Inference and Comparison with ARCH" Models.
  • Review of Economic Studies,
  • Kottas, A., Branco, M. D., and Gelfand, A. E. (2002). "A Nonparametric Bayesian Modeling Approach for Cytogenetic Dosimetry." Biometrics, 58: 593–600.
  • Laws, D. J. and O'Hagan, A. (2002). "A Hierarchical Bayes Model for Multilocation Auditing." Journal of the Royal Statistical Society, Series D, 51: 431–450.
  • Lo, A. Y. (1984). "On a class of Bayesian Nonparametric Estimates: I. Density Estimates." Annals of Statistics, 12: 351–357.
  • MacEachern, S. N. (1999). "Dependent Nonparametric Processes." In ASA Proceedings of the Section on Bayesian Statistical Science, 50–55.
  • –- (2000). "Dependent Dirichlet Processes." Technical report, Ohio State University, Department of Statistics.
  • Medvedovic, M. and Sivaganesan, S. (2002). "Bayesian Infinite Mixture Model-Based Clustering of Gene Expression Profiles." Bioinformatics, 18: 1194–1206.
  • Müller, P., Quintana, F., and Rosner, G. (2004). "Hierarchical Meta-Analysis over Related Non-parametric Bayesian Models." Journal of Royal Statistical Society, Series B, 66: 735–749.
  • Mulliere, P. and Tardella, L. (1998). "Approximating distributions of random functionals of Ferguson-Dirichlet priors." The Canadian Journal of Statistics, 26: 283–297.
  • Murray, I. and Adams, R. P. (2010). "Slice sampling covariance hyperparameters of latent Gaussian models." In Advances in Neural Information Processing Systems 23.
  • Murray, I., Adams, R. P., and MacKay, J. C. (2010). "Elliptical slice sampling." Journal of Machine Learning Research, 9: 541–548.
  • Ongaro, A. and Cattaneo, C. (2004). "Discrete random probability measures: a general framework for nonparametric Bayesian inference." Statistics and Probability Letters, 67: 33–45.
  • Orbanz, P. and Buhlmann, J. M. (2008). "Nonparametric Bayesian image segmentation." International Journal of Computer Vision, 77: 23–45.
  • Papaspiliopoulos, O. (2009). "A note on posterior sampling from Dirichlet process mixture models." Technical report, Department of Economics, Universitat Pompeu Fabra.
  • Pitman, J. (1995). "Exchangeable and Partially Exchangeable Random Partitions." Probability Theory and Related Fields, 102: 145–158.
  • –- (1996). "Some Developments of the Blackwell-MacQueen Urn Scheme." In Ferguson, T. S., Shapeley, L. S., and MacQueen, J. B. (eds.), Statistics, Probability and Game Theory. Papers in Honor of David Blackwell, 245–268. Hayward, CA:IMS.
  • Robert, C. P. and Casella, G. (1999). Monte Carlo Statistical Methods. Springer - Verlag.
  • Roberts, G. and Papaspiliopoulos, O. (2008). "Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models." Biometrika, 95: 169–186.
  • Rodriguez, A., Dunson, D. B., and Gelfand, A. E. (2008). "The nested Dirichlet process, with Discussion." Journal of American Statistical Association, 103: 1131–1154.
  • Rodriguez, A., Dunson, D. B., and Taylor, J. (2009). "Bayesian hierarchically weighted finite mixtures models for samples of distributions." Biostatistics, 10: 155–171.
  • Rodriguez, A. and Ter Horst, E. (2008). "Bayesian dynamic density estimation." Bayesian Analysis, 3: 339–366.
  • Rodriguez, A. and ter Horst, E. (2010). "Measuring expectations in options markets: An application to the S&P500 index." Quantitative Finance, doi:10.1080/ 14697680903193397.
  • Sethuraman, J. (1994). "A Constructive Definition of Dirichlet Priors." Statistica Sinica, 4: 639–650.
  • Sudderth, E. B. and Jordan, M. I. (2008). "Shared segmentation of natural scenes using dependent Pitman-Yor processes." In Proceedings of Neural Information Proceesing Systems 20.
  • Teh, Y. W., Jordan, M. I., Beal, M. J., and Blei, D. M. (2006). "Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes." Journal of the American Statistical Association, 101: 1566–1581.
  • Walker, S. G. (2007). "Sampling the Dirichlet mixture model with slices." Commnunications in Statistics, Part B - Simulation and Computation, 36: 45–54.
  • West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models. Springer - Verlag, New York, second edition edition.