The Annals of Statistics

Posterior consistency of Gaussian process prior for nonparametric binary regression

Subhashis Ghosal and Anindya Roy

Full-text: Open access


Consider binary observations whose response probability is an unknown smooth function of a set of covariates. Suppose that a prior on the response probability function is induced by a Gaussian process mapped to the unit interval through a link function. In this paper we study consistency of the resulting posterior distribution. If the covariance kernel has derivatives up to a desired order and the bandwidth parameter of the kernel is allowed to take arbitrarily small values, we show that the posterior distribution is consistent in the L1-distance. As an auxiliary result to our proofs, we show that, under certain conditions, a Gaussian process assigns positive probabilities to the uniform neighborhoods of a continuous function. This result may be of independent interest in the literature for small ball probabilities of Gaussian processes.

Article information

Ann. Statist., Volume 34, Number 5 (2006), 2413-2429.

First available in Project Euclid: 23 January 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Binary regression Gaussian process Karhunen–Loeve expansion maximal inequality posterior consistency reproducing kernel Hilbert space


Ghosal, Subhashis; Roy, Anindya. Posterior consistency of Gaussian process prior for nonparametric binary regression. Ann. Statist. 34 (2006), no. 5, 2413--2429. doi:10.1214/009053606000000795.

Export citation


  • Adler, R. J. (1990). An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes. IMS, Hayward, CA.
  • Choi, T. and Schervish, M. J. (2004). Posterior consistency in nonparametric regression problem under Gaussian process prior. Preprint.
  • Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050--1059.
  • Choudhuri, N., Ghosal, S. and Roy, A. (2007). Nonparametric binary regression using a Gaussian process prior. Stat. Methodol. 4. To appear. Available at
  • Cramér, H. and Leadbetter, M. R. (1967). Stationary and Related Stochastic Processes. Sample Function Properties and Their Applications. Wiley, New York.
  • Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143--158.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500--531.
  • Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13--30.
  • Lenk, P. J. (1988). The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 509--516.
  • Leonard, T. (1978). Density estimation, stochastic processes and prior information (with discussion). J. Roy. Statist. Soc. Ser. B 40 113--146.
  • Tokdar, S. T. and Ghosh, J. K. (2007). Posterior consistency of Gaussian process priors in density estimation. J. Statist. Plann. Inference 137 34--42.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York.
  • Walker, S. G. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028--2043.