Bernoulli

  • Bernoulli
  • Volume 25, Number 3 (2019), 2330-2358.

Bayesian mode and maximum estimation and accelerated rates of contraction

William Weimin Yoo and Subhashis Ghosal

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We study the problem of estimating the mode and maximum of an unknown regression function in the presence of noise. We adopt the Bayesian approach by using tensor-product B-splines and endowing the coefficients with Gaussian priors. In the usual fixed-in-advanced sampling plan, we establish posterior contraction rates for mode and maximum and show that they coincide with the minimax rates for this problem. To quantify estimation uncertainty, we construct credible sets for these two quantities that have high coverage probabilities with optimal sizes. If one is allowed to collect data sequentially, we further propose a Bayesian two-stage estimation procedure, where a second stage posterior is built based on samples collected within a credible set constructed from a first stage posterior. Under appropriate conditions on the radius of this credible set, we can accelerate optimal contraction rates from the fixed-in-advanced setting to the minimax sequential rates. A simulation experiment shows that our Bayesian two-stage procedure outperforms single-stage procedure and also slightly improves upon a non-Bayesian two-stage procedure.

Article information

Source
Bernoulli, Volume 25, Number 3 (2019), 2330-2358.

Dates
Received: August 2017
Revised: March 2018
First available in Project Euclid: 12 June 2019

Permanent link to this document
https://projecteuclid.org/euclid.bj/1560326447

Digital Object Identifier
doi:10.3150/18-BEJ1056

Mathematical Reviews number (MathSciNet)
MR3961250

Zentralblatt MATH identifier
07066259

Keywords
anisotropic Hölder space credible set maximum value mode nonparametric regression posterior contraction sequential tensor-product B-splines two-stage

Citation

Yoo, William Weimin; Ghosal, Subhashis. Bayesian mode and maximum estimation and accelerated rates of contraction. Bernoulli 25 (2019), no. 3, 2330--2358. doi:10.3150/18-BEJ1056. https://projecteuclid.org/euclid.bj/1560326447


Export citation

References

  • [1] Belitser, E., Ghosal, S. and van Zanten, H. (2012). Optimal two-stage procedures for estimating location and size of the maximum of a multivariate regression function. Ann. Statist. 40 2850–2876.
  • [2] Blum, J.R. (1954). Multidimensional stochastic approximation methods. Ann. Math. Statist. 25 737–744.
  • [3] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999–2028.
  • [4] Castillo, I. and Nickl, R. (2014). On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941–1969.
  • [5] Chen, H. (1988). Lower rate of convergence for locating a maximum of a function. Ann. Statist. 16 1330–1334.
  • [6] Cox, D.D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [7] de Boor, C. (2001). A Practical Guide to Splines, Revised ed. New York: Springer.
  • [8] Dippon, J. (2003). Accelerated randomized stochastic optimization. Ann. Statist. 31 1260–1281.
  • [9] Fabian, V. (1967). Stochastic approximation of minima with improved asymptotic speed. Ann. Math. Statist. 38 191–200.
  • [10] Facer, M.R. and Müller, H.-G. (2003). Nonparametric estimation of the location of a maximum in a response surface. J. Multivariate Anal. 87 191–217.
  • [11] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [12] Ghosal, S. and van der Vaart, A.W. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge: Cambridge Univ. Press.
  • [13] Hasminskiĭ, R.Z. (1979). Lower bound for the risks of nonparametric estimates of the mode. In Contributions to Statistics (J. Jureckova, ed.) 91–97. Dordrecht: Reidel.
  • [14] Jørgensen, M., Nielsen, C.T., Keiding, N. and Skakkeback, N.E. (1985). Parametrische und nichtparametrische Modelle für Wachstumsdaten. In Neuere Verfahren der nichtparametrischen Statistik (G.C. Pflug, ed.). Med. Informatik und Statistik 60 74–87. Berlin: Springer.
  • [15] Kiefer, J. and Wolfowitz, J. (1952). Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23 462–466.
  • [16] Kim, J. and Pollard, D. (1990). Cube root asymptotics. Ann. Statist. 18 191–219.
  • [17] Knapik, B.T., van der Vaart, A.W. and van Zanten, J.H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [18] Lan, Y., Banerjee, M. and Michailidis, G. (2009). Change-point estimation under adaptive sampling. Ann. Statist. 37 1752–1791.
  • [19] Mokkadem, A. and Pelletier, M. (2007). A companion for the Kiefer–Wolfowitz–Blum stochastic approximation algorithm. Ann. Statist. 35 1749–1772.
  • [20] Müller, H.-G. (1985). Kernel estimators of zeros and of location and size of extrema of regression functions. Scand. J. Stat. 12 221–232.
  • [21] Müller, H.-G. (1989). Adaptive nonparametric peak estimation. Ann. Statist. 17 1053–1069.
  • [22] Polyak, B.T. and Tsybakov, A.B. (1990). Optimal order of accuracy of search algorithms in stochastic optimization. Probl. Inf. Transm. 26 126–133.
  • [23] Schumaker, L.L. (2007). Spline Functions: Basic Theory, 3rd ed. New York: Cambridge Univ. Press.
  • [24] Shen, W. and Ghosal, S. (2015). Adaptive Bayesian procedures using random series priors. Scand. J. Statist. 42 1194–1213.
  • [25] Shoung, J.-M. and Zhang, C.-H. (2001). Least squares estimators of the mode of a unimodal regression function. Ann. Statist. 29 648–665.
  • [26] Silverman, B.W. (1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with discussion). J. Roy. Statist. Soc. Ser. B 47 1–52.
  • [27] Szabó, B., van der Vaart, A.W. and van Zanten, J.H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428.
  • [28] Tang, R., Banerjee, M. and Michailidis, G. (2011). A two-stage hybrid procedure for estimating an inverse regression function. Ann. Statist. 39 956–989.
  • [29] Tsybakov, A.B. (1990). Recursive estimation of the mode of a multivariate distribution. Probl. Inf. Transm. 26 31–37.
  • [30] van der Vaart, A.W. (1998). Asymptotic Statistics. Cambridge: Cambridge Univ. Press.
  • [31] van der Vaart, A.W. and van Zanten, J.H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. (IMS) Collect. 3 200–222. Beachwood, OH: IMS.
  • [32] van der Vaart, A.W. and Wellner, J.A. (1996). Weak Convergence and Empirical Processes With Applications to Statistics. New York: Springer.
  • [33] Yoo, W.W. and Ghosal, S. (2016). Supremum norm posterior contraction and credible sets for nonparametric multivariate regression. Ann. Statist. 44 1069–1102.
  • [34] Yoo, W.W. and Ghosal, S. (2018). Supplement to “Bayesian mode and maximum estimation and accelerated rates of contraction.” DOI:10.3150/18-BEJ1056SUPP.
  • [35] Yoo, W.W., Rivoirard, V. and Rousseau, J. (2018). Adaptive supremum norm posterior contraction: Wavelet spike-and-slab and anisotropic Besov spaces. arXiv:1708.01909 [math.ST].
  • [36] Yoo, W.W. and van der Vaart, A.W. (2018). The Bayes Lepski’s method and credible bands through volume of tubular neighborhoods. arXiv:1711.06926 [math.ST].

Supplemental materials

  • Supplement to “Bayesian mode and maximum estimation and accelerated rates of contraction”. The supplementary file contains detailed proofs of Corollary 4.2, Proposition 5.1 and Corollary 8.4.