Annals of Statistics

Bayesian inverse problems with Gaussian priors

B. T. Knapik, A. W. van der Vaart, and J. H. van Zanten

Full-text: Open access


The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive.

Article information

Ann. Statist., Volume 39, Number 5 (2011), 2626-2657.

First available in Project Euclid: 22 December 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G15: Tolerance and confidence regions
Secondary: 62G20: Asymptotic properties

Credible set posterior distribution Gaussian prior rate of contraction


Knapik, B. T.; van der Vaart, A. W.; van Zanten, J. H. Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 (2011), no. 5, 2626--2657. doi:10.1214/11-AOS920.

Export citation


  • [1] Bertero, M. (1986). Regularization methods for linear inverse problems. In Inverse Problems (Montecatini Terme, 1986). Lecture Notes in Math. 1225 52–112. Springer, Berlin.
  • [2] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Univ. Press, Baltimore, MD.
  • [3] Bissantz, N., Hohage, T., Munk, A. and Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J. Numer. Anal. 45 2610–2636 (electronic).
  • [4] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.
  • [5] Castillo, I. (2011). A semiparametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields. To appear. DOI:10.1007/s00440-010-0316-5.
  • [6] Cavalier, L. (2008). Nonparametric statistical inverse problems. Inverse Problems 24 034004, 19.
  • [7] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [8] Donoho, D. L. (1994). Statistical estimation and optimal recovery. Ann. Statist. 22 238–270.
  • [9] Donoho, D. L. and Low, M. G. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944–970.
  • [10] Feller, W. (1971). An Introduction to Probability Theory and Its Applications. Vol. II, 2nd ed. Wiley, New York.
  • [11] Florens, J. and Simoni, A. (2011). Regularizing priors for linear inverse problems. Preprint.
  • [12] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [13] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [14] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • [15] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [16] Goldenshluger, A. and Pereverzev, S. V. (2003). On adaptive inverse estimation of linear functionals in Hilbert scales. Bernoulli 9 783–807.
  • [17] Halmos, P. R. (1967). A Hilbert Space Problem Book. Van Nostrand, Princeton.
  • [18] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [19] Mair, B. A. and Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM J. Appl. Math. 56 1424–1444.
  • [20] Mandelbaum, A. (1984). Linear estimators and measurable linear transformations on a Hilbert space. Z. Wahrsch. Verw. Gebiete 65 385–397.
  • [21] Mathé, P. and Pereverzev, S. V. (2006). Regularization of some linear ill-posed problems with discretized random noisy data. Math. Comp. 75 1913–1929 (electronic).
  • [22] Nashed, M. Z. and Wahba, G. (1974). Generalized inverses in reproducing kernel spaces: An approach to regularization of linear operator equations. SIAM J. Math. Anal. 5 974–987.
  • [23] Rudin, W. (1991). Functional Analysis, 2nd ed. McGraw-Hill, New York.
  • [24] Ruymgaart, F. H. (1993). A unified approach to inversion problems in statistics. Math. Methods Statist. 2 130–146.
  • [25] Skorohod, A. V. (1974). Integration in Hilbert Space. Ergebnisse der Mathematik und Ihrer Grenzgebiete 79. Springer, New York. Translated from the Russian by Kenneth Wickwire.
  • [26] Stuart, A. M. (2010). Inverse problems: A Bayesian perspective. Acta Numer. 19 451–559.
  • [27] Tikhonov, A. N. and Arsenin, V. Y. (1977). Solutions of Ill-posed Problems. V. H. Winston and Sons, Washington, DC. Translated from the Russian, Preface by translation editor Fritz John, Scripta Series in Mathematics.
  • [28] van der Vaart, A. (1991). On differentiable functionals. Ann. Statist. 19 178–204.
  • [29] van der Vaart, A. and van Zanten, H. (2007). Bayesian inference with rescaled Gaussian process priors. Electron. J. Stat. 1 433–448 (electronic).
  • [30] van der Vaart, A. W. (1988). Statistical Estimation in Large Parameter Spaces. CWI Tract 44. Stichting Mathematisch Centrum Centrum voor Wiskunde en Informatica, Amsterdam.
  • [31] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [32] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.
  • [33] van Rooij, A. C. M. and Ruymgaart, F. H. (2001). Abstract inverse estimation with application to deconvolution on locally compact abelian groups. Ann. Inst. Statist. Math. 53 781–798.