Electronic Journal of Statistics

Bayesian inverse problems with non-conjugate priors

Kolyan Ray

Full-text: Open access


We investigate the frequentist posterior contraction rate of nonparametric Bayesian procedures in linear inverse problems in both the mildly and severely ill-posed cases. A theorem is proved in a general Hilbert space setting under approximation-theoretic assumptions on the prior. The result is applied to non-conjugate priors, notably sieve and wavelet series priors, as well as in the conjugate setting. In the mildly ill-posed setting minimax optimal rates are obtained, with sieve priors being rate adaptive over Sobolev classes. In the severely ill-posed setting, oversmoothing the prior yields minimax rates. Previously established results in the conjugate setting are obtained using this method. Examples of applications include deconvolution, recovering the initial condition in the heat equation and the Radon transform.

Article information

Electron. J. Statist., Volume 7 (2013), 2516-2549.

First available in Project Euclid: 8 October 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G20: Asymptotic properties
Secondary: 62G05: Estimation 62G08: Nonparametric regression

Rate of contraction posterior distribution nonparametric hypothesis testing


Ray, Kolyan. Bayesian inverse problems with non-conjugate priors. Electron. J. Statist. 7 (2013), 2516--2549. doi:10.1214/13-EJS851. https://projecteuclid.org/euclid.ejs/1381239960

Export citation


  • [1] Agapiou, S., Larsson, S. and Stuart, A. M. (2013). Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems., Stochastic Process. Appl. 123 3828–3860.
  • [2] Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior., Scand. J. Stat. 40 549–570.
  • [3] Bissantz, N., Hohage, T., Munk, A. and Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications., SIAM J. Numer. Anal. 45 2610–2636.
  • [4] Borell, C. (1975). The Brunn-Minkowski inequality in Gauss space., Invent. Math. 30 207–216.
  • [5] Butucea, C. and Tsybakov, A. B. (2007). Sharp optimality in density deconvolution with dominating bias. II., Teor. Veroyatn. Primen. 52 336–349.
  • [6] Castillo, I., Kerkyacharian, G. and Picard, D. (2013). Thomas Bayes’ walk on manifolds., Probab. Theory Related Fields. To appear.
  • [7] Cavalier, L. (2008). Nonparametric statistical inverse problems., Inverse Problems 24 034004, 19.
  • [8] Da Prato, G. (2006)., An introduction to infinite-dimensional analysis. Universitext. Springer-Verlag, Berlin. Revised and extended from the 2001 original by Da Prato.
  • [9] Dembo, A., Mayer-Wolf, E. and Zeitouni, O. (1995). Exact behavior of Gaussian seminorms., Statist. Probab. Lett. 23 275–280.
  • [10] Folland, G. B. (1999)., Real analysis, second ed. Pure and Applied Mathematics (New York). John Wiley & Sons Inc., New York. Modern techniques and their applications, A Wiley-Interscience Publication.
  • [11] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions., Ann. Statist. 28 500–531.
  • [12] Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities., Ann. Statist. 35 697–723.
  • [13] Giné, E. and Nickl, R. (2011). Rates on contraction for posterior distributions in $L^r$-metrics, $1\leq r\leq\infty$., Ann. Statist. 39 2883–2911.
  • [14] Hoffmann-Jørgensen, J., Shepp, L. A. and Dudley, R. M. (1979). On the lower tail of Gaussian seminorms., Ann. Probab. 7 319–342.
  • [15] Holmes, C. and Denison, G. (1999). Bayesian wavelet analysis with a model complexity prior. In, Bayesian Statistics 6: proceedings of the sixth Valencia international meeting 769–776. Clarendon Press, Oxford.
  • [16] Huang, T.-M. (2004). Convergence rates for posterior distributions and adaptive estimation., Ann. Statist. 32 1556–1593.
  • [17] Johnstone, I. M., Kerkyacharian, G., Picard, D. and Raimondo, M. (2004). Wavelet deconvolution in a periodic setting., J. R. Stat. Soc. Ser. B Stat. Methodol. 66 547–573.
  • [18] Johnstone, I. M. and Silverman, B. W. (1990). Speed of estimation in positron emission tomography and related inverse problems., Ann. Statist. 18 251–280.
  • [19] Knapik, B. T., Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2012). Bayes procedures for adaptive inference in inverse problems for the white noise model., arXiv:1209.3628.
  • [20] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors., Ann. Statist. 39 2626–2657.
  • [21] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2013). Bayesian recovery of the initial condition for the heat equation., Comm. Statist. Theory Methods 42 1294–1313.
  • [22] Kuelbs, J. and Li, W. V. (1993). Metric entropy and the small ball problem for Gaussian measures., J. Funct. Anal. 116 133–157.
  • [23] Ledoux, M. (2001)., The concentration of measure phenomenon. Mathematical Surveys and Monographs 89. American Mathematical Society, Providence, RI.
  • [24] Lounici, K. and Nickl, R. (2011). Global uniform risk bounds for wavelet deconvolution estimators., Ann. Statist. 39 201–231.
  • [25] Meyer, Y. (1992)., Wavelets and operators. Cambridge Studies in Advanced Mathematics 37. Cambridge University Press, Cambridge. Translated from the 1990 French original by D. H. Salinger.
  • [26] Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models., J. Comput. Graph. Statist. 9 249–265.
  • [27] Pensky, M. and Vidakovic, B. (1999). Adaptive wavelet estimator for nonparametric density deconvolution., Ann. Statist. 27 2033–2053.
  • [28] Rudin, W. (1991)., Functional analysis, second ed. International Series in Pure and Applied Mathematics. McGraw-Hill Inc., New York.
  • [29] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions., Ann. Statist. 29 687–714.
  • [30] Sytaya, G. N. (1974). On some asymptotic representations of the Gaussian measure in a Hilbert space., Theory of Stochastic Processes, Ukrainian Academy of Sciences, 2 93–104.
  • [31] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist. 36 1435–1463.
  • [32] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In, Pushing the limits of contemporary statistics: contributions in honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. Inst. Math. Statist., Beachwood, OH.
  • [33] van der Vaart, A. W. and van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth., Ann. Statist. 37 2655–2675.
  • [34] Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems., Ann. Statist. 28 532–552.