Electronic Journal of Statistics

Functional regression via variational Bayes

Jeff Goldsmith, Matt P. Wand, and Ciprian Crainiceanu

Full-text: Open access


We introduce variational Bayes methods for fast approximate inference in functional regression analysis. Both the standard cross-sectional and the increasingly common longitudinal settings are treated. The methodology allows Bayesian functional regression analyses to be conducted without the computational overhead of Monte Carlo methods. Confidence intervals of the model parameters are obtained both using the approximate variational approach and nonparametric resampling of clusters. The latter approach is possible because our variational Bayes functional regression approach is computationally efficient. A simulation study indicates that variational Bayes is highly accurate in estimating the parameters of interest and in approximating the Markov chain Monte Carlo-sampled joint posterior distribution of the model parameters. The methods apply generally, but are motivated by a longitudinal neuroimaging study of multiple sclerosis patients. Code used in simulations is made available as a web-supplement.

Article information

Electron. J. Statist. Volume 5 (2011), 572-602.

First available in Project Euclid: 15 June 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Approximate Bayesian inference Markov chain Monte Carlo penalized splines


Goldsmith, Jeff; Wand, Matt P.; Crainiceanu, Ciprian. Functional regression via variational Bayes. Electron. J. Statist. 5 (2011), 572--602. doi:10.1214/11-EJS619. http://projecteuclid.org/euclid.ejs/1308143123.

Export citation


  • [1] Basser, P., Mattiello, J. and LeBihan, D. (1994). MR Diffusion Tensor Spectroscopy and Imaging., Biophysical Journal 66 259-267.
  • [2] Basser, P., Pajevic, S., Pierpaoli, C. and Duda, J. (2000). In vivo fiber tractography using DT-MRI data., Magnetic Resonance in Medicine 44 625-632.
  • [3] Bishop, C. M. (2006)., Pattern Recognition and Machine Learning. New York: Springer.
  • [4] Cardot, H., Ferraty, F. and Sarda, P. (1999). Functional Linear Model., Statistics and Probability Letters 45 11-22.
  • [5] Crainiceanu, C. M. and Goldsmith, J. (2010). Bayesian Functional Data Analysis using WinBUGS., Journal of Statistical Software 32 1-33.
  • [6] Goldsmith, J., Bobb, J., Crainiceanu, C. M., Caffo, B. and Reich, D. (To Appear). Penalized Functional Regression., Journal of Computational and Graphical Statistics.
  • [7] Goldsmith, J., Crainiceanu, C. M., Caffo, B. and Reich, D. (2011). A Case Study of Longitudinal Association Between Disability and Neuronal Tract Measurements., Under Review.
  • [8] Gronwall, D. M. A. (1977). Paced auditory serial-addition task: A measure of recovery from concussion., Perceptual and Motor Skills 44 367-373.
  • [9] James, G. M., J, W. and J, Z. (2009). Functional Linear Regression That’s Interpretable., Annals of Statistics 37 2083-2108.
  • [10] Jordan, M. I. (2004). Graphical models., Statistical Science 19 140-155.
  • [11] Jordan, M. I., Ghahramani, Z., Jaakkola, T. S. and Saul, L. K. (1999). An Introduction to Variational Methods for Graphical Models., Machine Learning 37 183-233.
  • [12] Kullback, S. and Leibler, D. (1951). On Information and Sufficiency., The Annals of Mathematical Statistics 22 79-86.
  • [13] Lang, S. and Brezger, A. (2004). Bayesian P-splines., Journal of Computational and Graphical Statistics 13 183-212.
  • [14] LeBihan, D., Mangin, J., Poupon, C. and Clark, C. (2001). Diffusion Tensor Imaging: Concepts and Applications., Journal of Magnetic Resonance Imaging 13 534-546.
  • [15] Lin, X., Tench, C. R., Morgan, P. S. and Constantinescu, C. S. (2008). Use of combined conventional and quantitative MRI to quantify pathology related to cognitive impairment in multiple sclerosis., Journal of Neurology, Neurosurgery, and Psychiatry 237 437-441.
  • [16] McGrory, C. A., Titterington, D. M., Reeves, R. and Pettitt, A. N. (2009). Variational Bayes for Estimating the Parameters of a Hidden Potts Model., Statistics and Computing 19 329-340.
  • [17] Mori, S. and Barker, P. (1999). Diffusion magnetic resonance imaging: its principle and applications., The Anatomical Record 257 102-109.
  • [18] Müller, H.-G. and Stadtmüller, U. (2005). Generalized functional linear models., Annals of Statistics 33 774-805.
  • [19] Ormerod, J. and Wand, M. P. (2010). Explaining Variational Approximations., The American Statistician 64 140-153.
  • [20] Ozturk, A., Smith, S., Gordon-Lipkin, E., Harrison, D., Shiee, N., Pham, D., Caffo, B., Calabresi, P. and Reich, D. (2010). MRI of the corpus callosum in multiple sclerosis: association with disability., Multiple Sclerosis 16 166–177.
  • [21] Ramsay, J. O. and Silverman, B. W. (2005)., Functional Data Analysis. New York: Springer.
  • [22] Reiss, P. and Ogden, R. (2007). Functional Principal Component Regression and Functional Partial Least Squares., Journal of the American Statistical Association 102 984-996.
  • [23] Ruppert, D., Wand, M. P. and Carroll, R. J. (2003)., Semiparametric Regression. Cambridge: Cambridge University Press.
  • [24] Teschendorff, A. E., Wang, Y., Barbosa-Morais, N. L., Brenton, J. D. and Caldas, C. (2005). A Variational Bayesian Mixture Modeling Framework for Cluster Analysis of Gene-Expression Data., Bioinformatics 21 3025-3033.
  • [25] Titterington, D. M. (2004). Bayesian Methods for Neural Networks and Related Models., Statistical Science 19 128-139.