The Annals of Statistics

Convergence rates of posterior distributions for noniid observations

Subhashis Ghosal and Aad van der Vaart

Full-text: Open access

Abstract

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model.

Article information

Source
Ann. Statist., Volume 35, Number 1 (2007), 192-223.

Dates
First available in Project Euclid: 6 June 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1181100186

Digital Object Identifier
doi:10.1214/009053606000001172

Mathematical Reviews number (MathSciNet)
MR2332274

Zentralblatt MATH identifier
1114.62060

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G08: Nonparametric regression

Keywords
Covering numbers Hellinger distance independent nonidentically distributed observations infinite dimensional model Markov chains posterior distribution rate of convergence tests

Citation

Ghosal, Subhashis; van der Vaart, Aad. Convergence rates of posterior distributions for noniid observations. Ann. Statist. 35 (2007), no. 1, 192--223. doi:10.1214/009053606000001172. https://projecteuclid.org/euclid.aos/1181100186


Export citation

References

  • Amewou-Atisso, M., Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (2003). Posterior consistency for semiparametric regression problems. Bernoulli 9 291--312.
  • Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536--561.
  • Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrsch. Verw. Gebiete 65 181--237.
  • Birgé, L. (1983). Robust testing for independent non-identically distributed variables and Markov chains. In Specifying Statistical Models. From Parametric to Non-Parametric. Using Bayesian or Non-Bayesian Approaches. Lecture Notes in Statist. 16 134--162. Springer, New York.
  • Birgé, L. (2006). Model selection via testing: An alternative to (penalized) maximum likelihood estimators. Ann. Inst. H. Poincaré Probab. Statist. 42 273--325
  • Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Methods, 2nd ed. Springer, New York.
  • Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050--1059.
  • Choudhuri, N., Ghosal, S. and Roy, A. (2004). Contiguity of the Whittle measure for a Gaussian time series. Biometrika 91 211--218.
  • Chow, Y. S. and Teicher, H. (1978). Probability Theory. Independence, Interchangeability, Martingales. Springer, New York.
  • Dahlhaus, R. (1988). Empirical spectral processes and their applications to time series analysis. Stochastic Process. Appl. 30 69--83.
  • de Boor, C. (1978). A Practical Guide to Splines. Springer, New York.
  • Ghosal, S. (2001). Convergence rates for density estimation with Bernstein polynomials. Ann. Statist. 29 1264--1280.
  • Ghosal, S., Ghosh, J. K. and Samanta, T. (1995). On convergence of posterior distributions. Ann. Statist. 23 2145--2152.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500--531.
  • Ghosal, S., Lember, J. and van der Vaart, A. W. (2003). On Bayesian adaptation. Acta Appl. Math. 79 165--175.
  • Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233--1263.
  • Ghosal, S. and van der Vaart, A. W. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35. To appear. Available at www4.stat.ncsu.edu/~sghosal/papers.html.
  • Ibragimov, I. A. (1962). Some limit theorems for stationary processes. Theory Probab. Appl. 7 349--382.
  • Ibragimov, I. A. and Has'minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York.
  • Le Cam, L. M. (1973). Convergence of estimates under dimensionality restrictions. Ann. Statist. 1 38--53.
  • Le Cam, L. M. (1975). On local and global properties in the theory of asymptotic normality of experiments. In Stochastic Processes and Related Topics (M. L. Puri, ed.) 13--54. Academic Press, New York.
  • Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, New York.
  • Petrone, S. (1999). Bayesian density estimation using Bernstein polynomials. Canad. J. Statist. 27 105--126.
  • Pollard, D. (1990). Empirical Processes: Theory and Applications. IMS, Hayward, CA.
  • Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222--235.
  • Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687--714.
  • Stone, C. J. (1990). Large-sample inference for log-spline models. Ann. Statist. 18 717--741.
  • Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118--184.
  • van der Meulen, F., van der Vaart, A. W. and van Zanten, J. H. (2006). Convergence rates of posterior distributions for Brownian semimartingale models. Bernoulli 12 863--888.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York.
  • Whittle, P. (1957). Curve and periodogram smoothing (with discussion). J. Roy. Statist. Soc. Ser. B 19 38--63.
  • Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339--362.
  • Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532--552.