Bayesian Analysis

On Bayesian A- and D-Optimal Experimental Designs in Infinite Dimensions

Alen Alexanderian, Philip J. Gloor, and Omar Ghattas

Full-text: Open access

Abstract

We consider Bayesian linear inverse problems in infinite-dimensional separable Hilbert spaces, with a Gaussian prior measure and additive Gaussian noise model, and provide an extension of the concept of Bayesian D-optimality to the infinite-dimensional case. To this end, we derive the infinite-dimensional version of the expression for the Kullback–Leibler divergence from the posterior measure to the prior measure, which is subsequently used to derive the expression for the expected information gain. We also study the notion of Bayesian A-optimality in the infinite-dimensional setting, and extend the well known (in the finite-dimensional case) equivalence of the Bayes risk of the MAP estimator with the trace of the posterior covariance, for the Gaussian linear case, to the infinite-dimensional Hilbert space case.

Article information

Source
Bayesian Anal., Volume 11, Number 3 (2016), 671-695.

Dates
First available in Project Euclid: 26 August 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1440594948

Digital Object Identifier
doi:10.1214/15-BA969

Mathematical Reviews number (MathSciNet)
MR3498042

Zentralblatt MATH identifier
1359.62315

Keywords
Bayesian inference in Hilbert space Gaussian measure Kullback–Leibler divergence Bayesian optimal experimental design expected information gain Bayes risk

Citation

Alexanderian, Alen; Gloor, Philip J.; Ghattas, Omar. On Bayesian A- and D-Optimal Experimental Designs in Infinite Dimensions. Bayesian Anal. 11 (2016), no. 3, 671--695. doi:10.1214/15-BA969. https://projecteuclid.org/euclid.ba/1440594948


Export citation

References

  • Alexanderian, A., Petra, N., Stadler, G., and Ghattas, O. (2014). “A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized $\ell_{0}$-sparsification.” SIAM Journal on Scientific Computing, 36(5): A2122–A2148.
  • Atkinson, A. C. and Donev, A. N. (1992). Optimum Experimental Designs. Oxford.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer.
  • Bui-Thanh, T., Ghattas, O., Martin, J., and Stadler, G. (2013). “A computational framework for infinite-dimensional Bayesian inverse problems Part I: The linearized case, with application to global seismic inversion.” SIAM Journal on Scientific Computing, 35(6): A2494–A2523.
  • Carlin, B. P. and Louis, T. A. (1997). “Bayes and empirical Bayes methods for data analysis.” Statistics and Computing, 7(2): 153–154.
  • Chaloner, K. and Verdinelli, I. (1995). “Bayesian experimental design: A review.” Statistical Science, 10(3): 273–304.
  • Conway, J. B. (2000). A Course in Operator Theory. American Mathematical Society.
  • Da Prato, G. (2006). An Introduction to Infinite-Dimensional Analysis. Springer.
  • Da Prato, G. and Zabczyk, J. (2002). Second Order Partial Differential Equations in Hilbert Spaces, volume 293. Cambridge University Press.
  • — (2014). Stochastic Equations in Infinite Dimensions, volume 152. Cambridge University Press.
  • Dashti, M., Law, K. J., Stuart, A. M., and Voss, J. (2013). “MAP estimators and their consistency in Bayesian nonparametric inverse problems.” Inverse Problems, 29.
  • Flath, H. P., Wilcox, L. C., Akçelik, V., Hill, J., van Bloemen Waanders, B., and Ghattas, O. (2011). “Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations.” SIAM Journal on Scientific Computing, 33(1).
  • Gel’fand, I. M. and Vilenkin, N. Y. (1964). Generalized Functions. Vol. 4: Applications of Harmonic Analysis. Translated by Amiel Feinstein. Academic Press, New York - London, 1964.
  • Kullback, S. and Leibler, R. A. (1951). “On Information and Sufficiency.” Annals of Mathematical Statistics, 22(1): 79–86.
  • Lindgren, F., Rue, H., and Lindström, J. (2011). “An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(4): 423–498.
  • Pukelsheim, F. (2006). Optimal Design of Experiments, volume 50. SIAM.
  • Reed, M. and Simon, B. (1972). Methods of Modern Mathematical Physics. Vol. 1.: Functional Analysis. Academic Press.
  • Simon, B. (1977). “Notes on Infinite Determinants of Hilbert Space Operators.” Advances in Mathematics, 24: 244–273.
  • Stuart, A. M. (2010). “Inverse problems: a Bayesian perspective.” Acta Numerica, 19: 451–559.
  • Tarantola, A. (2005). Inverse problem theory and methods for model parameter estimation. siam.
  • Uciński, D. (2005). Optimal Measurement Methods for Distributed Parameter System Identification. Boca Raton: CRC Press.