Bayesian Analysis

Smoothing and Mean–Covariance Estimation of Functional Data with a Bayesian Hierarchical Model

Jingjing Yang, Hongxiao Zhu, Taeryon Choi, and Dennis D. Cox

Full-text: Open access


Functional data, with basic observational units being functions (e.g., curves, surfaces) varying over a continuum, are frequently encountered in various applications. While many statistical tools have been developed for functional data analysis, the issue of smoothing all functional observations simultaneously is less studied. Existing methods often focus on smoothing each individual function separately, at the risk of removing important systematic patterns common across functions. We propose a nonparametric Bayesian approach to smooth all functional observations simultaneously and nonparametrically. In the proposed approach, we assume that the functional observations are independent Gaussian processes subject to a common level of measurement errors, enabling the borrowing of strength across all observations. Unlike most Gaussian process regression models that rely on pre-specified structures for the covariance kernel, we adopt a hierarchical framework by assuming a Gaussian process prior for the mean function and an Inverse-Wishart process prior for the covariance function. These prior assumptions induce an automatic mean–covariance estimation in the posterior inference in addition to the simultaneous smoothing of all observations. Such a hierarchical framework is flexible enough to incorporate functional data with different characteristics, including data measured on either common or uncommon grids, and data with either stationary or nonstationary covariance structures. Simulations and real data analysis demonstrate that, in comparison with alternative methods, the proposed Bayesian approach achieves better smoothing accuracy and comparable mean–covariance estimation results. Furthermore, it can successfully retain the systematic patterns in the functional observations that are usually neglected by the existing functional data analyses based on individual-curve smoothing.

Article information

Bayesian Anal., Volume 11, Number 3 (2016), 649-670.

First available in Project Euclid: 26 August 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

functional data smoothing Bayesian hierarchical model Gaussian process Matérn covariance function empirical Bayes


Yang, Jingjing; Zhu, Hongxiao; Choi, Taeryon; Cox, Dennis D. Smoothing and Mean–Covariance Estimation of Functional Data with a Bayesian Hierarchical Model. Bayesian Anal. 11 (2016), no. 3, 649--670. doi:10.1214/15-BA967.

Export citation


  • Banerjee, A., Dunson, D. B., and Tokdar, S. T. (2013). “Efficient Gaussian process regression for large datasets.” Biometrika, 100(1): 75–89.
  • Banerjee, S., Carlin, B. P., and Gelfand, A. E. (2014). Hierarchical Modeling and Analysis for Spatial Data. CRC Press.
  • Banerjee, S., Gelfand, A. E., Finley, A. O., and Sang, H. (2008). “Gaussian predictive process models for large spatial data sets.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(4): 825–848.
  • Buys, T. P., Cantor, S. B., Guillaud, M., Adler-Storthz, K., Cox, D. D., Okolo, C., Arulogon, O., Oladepo, O., Basen-Engquist, K., Shinn, E., et al. (2012). “Optical technologies and molecular imaging for cervical neoplasia: a program project update.” Gender Medicine, 9(1): S7–S24.
  • Cardot, H., Ferraty, F., and Sarda, P. (2003). “Spline estimators for the functional linear model.” Statistica Sinica, 13(3): 571–592.
  • Dawid, A. P. (1981). “Some matrix-variate distribution theory: notational considerations and a Bayesian application.” Biometrika, 68(1): 265–274.
  • Gelman, A. and Rubin, D. B. (1992). “Inference from iterative simulation using multiple sequences.” Statistical Science, 457–472.
  • Hall, P., Poskitt, D. S., and Presnell, B. (2001). “A Functional Data-Analytic Approach to Signal Discrimination.” Technometrics, 43(1): 1–9.
  • Hitchcock, D. B., Casella, G., and Booth, J. G. (2006). “Improved estimation of dissimilarities by presmoothing functional data.” Journal of the American Statistical Association, 101(473): 211–222.
  • Kaufman, C. G., Sain, S. R., et al. (2010). “Bayesian functional ANOVA modeling using Gaussian process prior distributions.” Bayesian Analysis, 5(1): 123–149.
  • Leng, X. and Müller, H.-G. (2006). “Classification using functional data analysis for temporal gene expression data.” Bioinformatics, 22: 68–76.
  • Müller, H.-G. (2005). “Functional modeling and classification of longitudinal data.” Scandinavian Journal of Statistics, 32: 223–240.
  • Nguyen, X. and Gelfand, A. E. (2014). “Bayesian nonparametric modeling for functional analysis of variance.” Annals of the Institute of Statistical Mathematics, 66(3): 495–526.
  • Quiñonero Candela, J., E., R. C., and Williams, C. K. I. (2007). “Approximation Methods for Gaussian Process Regression.” Technical report, Applied Games, Microsoft Research Ltd.
  • R Core Team (2013). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
  • Ramsay, J. O. and Dalzell, C. (1991). “Some tools for functional data analysis.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 539–572.
  • Ramsay, J. O. and Silverman, B. W. (2002). Applied Functional Data Analysis: Methods and Case Studies, volume 77. Springer, New York.
  • — (2005). Functional Data Analysis. Springer Series in Statistics. Springer, New York, second edition.
  • Rasmussen, C. E. and Williams, C. K. I. (2006). Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA.
  • Rice, J. A. and Silverman, B. W. (1991). “Estimating the Mean and Covariance Structure Nonparametrically When the Data Are Curves.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 53: 233–243.
  • Särkkä, S. and Aki, V. (2014). “MCMC Diagnostics for Matlab.”
  • Shi, J. Q. and Choi, T. (2011). Gaussian Process Regression Analysis for Functional Data. CRC Press, Boca Raton, FL.
  • Shi, J. Q., Wang, B., Will, E. J., and West, R. M. (2012). “Mixed-effects Gaussian process functional regression models with application to dose-response curve prediction.” Statistics in Medicine, 31(26): 3165–3177.
  • Stein, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer.
  • Von Neumann, J. (1941). “Distribution of the ratio of the mean square successive difference to the variance.” The Annals of Mathematical Statistics, 12(4): 367–395.
  • Wang, B. and Shi, J. Q. (2014). “Generalized Gaussian process regression model for non-Gaussian functional data.” Journal of the American Statistical Association, 109(507): 1123–1133.
  • Yamal, J.-M., Zewdie, G. A., Cox, D. D., Atkinson, E. N., Cantor, S. B., MacAulay, C., Davies, K., Adewole, I., Buys, T. P., and Follen, M. (2012). “Accuracy of optical spectroscopy for the detection of cervical intraepithelial neoplasia without colposcopic tissue information; a step toward automation for low resource settings.” Journal of Biomedical Optics, 17(4): 047002–047002.
  • Yao, F., Müller, H.-G., and Wang, J.-L. (2005a). “Functional Data Analysis for Sparse Longitudinal Data.” Journal of the American Statistical Association, 100(470): 577–590.
  • — (2005b). “Functional linear regression analysis for longitudinal data.” The Annals of Statistics, 33(6): 2873–2903.
  • Zhang, H. (2004). “Inconsistent estimation and asymptotically equal interpolations in model-based geostatistics.” Journal of the American Statistical Association, 99(465): 250–261.
  • Zhu, H. and Cox, D. D. (2009). “A functional generalized linear model with curve selection in cervical pre-cancer diagnosis using fluorescence spectroscopy.” In Optimality, volume 57 of IMS Lecture Notes Monograph Series, 173–189. Institute of Mathematical Statistics, Beachwood, OH.
  • Zhu, H., Strawn, N., and Dunson, B. D. (2014). “Bayesian graphical models for multivariate functional data.” arXiv:1411.4158
  • Zhu, H., Vannucci, M., and Cox, D. D. (2010). “A Bayesian Hierarchical Model for Classification with Selection of Functional Predictors.” Biometrics, 66: 463–473.