Bayesian Analysis

Reduced rank regression models with latent variables in Bayesian functional data analysis

Angelika van der Linde

Full-text: Open access


In functional data analysis (FDA) it is of interest to generalize techniques of multivariate analysis like canonical correlation analysis or regression to functions which are often observed with noise. In the proposed Bayesian approach to FDA two tools are combined: (i) a special Demmler-Reinsch like basis of interpolation splines to represent functions parsimoniously and flexibly; (ii) latent variable models initially introduced for probabilistic principal components analysis or canonical correlation analysis of the corresponding coefficients. In this way partial curves and non-Gaussian measurement error schemes can be handled. Bayesian inference is based on a variational algorithm such that computations are straight forward and fast corresponding to an idea of FDA as a toolbox for explorative data analysis. The performance of the approach is illustrated with synthetic and real data sets.

Article information

Bayesian Anal. Volume 6, Number 1 (2011), 77-126.

First available in Project Euclid: 13 June 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F15: Bayesian inference
Secondary: 62G99: None of the above, but in this section 62H20: Measures of association (correlation, canonical correlation, etc.) 62H25: Factor analysis and principal components; correspondence analysis 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]

Functional data analysis Functional canonical correlation analysis Functional regression Functional prediction Functional discriminant analysis


van der Linde, Angelika. Reduced rank regression models with latent variables in Bayesian functional data analysis. Bayesian Anal. 6 (2011), no. 1, 77--126. doi:10.1214/11-BA603.

Export citation


  • Amato, U., Antoniadis, A., and Feis, I. D. (2006). "Dimension Reduction in Functional Regression with Applications." Computational Statistics and Data Analalysis, 50: 2422–2446.
  • Archambeau, C., Delannay, N., and Verlysen, M. (2006). "Robust Probabilistic Projections." In Proceedings of the 23rd international conference on Machine Learning, ACM International Conference Proceeding Series, volume 148, 33 – 40.
  • Attias, H. (1999). "Inferring Parameters and Structure of Latent Variable Models by Variational Bayes." In Laskey, K. and Prade, H. (eds.), UAI '99: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, 21–30.
  • Bach, F. and Jordan, M. (2005). "A probabilistic interpretation of canonical correlation analysis." Technical Report 688. Dept. of Statistics, University of California, Berkeley.
  • Baladandayuthapani, V., Mallick, B., Hong, M., Lupton, J., Turner, N., and Carroll, R. (2007). "Bayesian Hierarchical Spatially Correlated Functional Data Analysis with Application to Colon Carcinogesis." Biometrics, 64: 64–73.
  • Bigelow, J. and Dunson, D. (2006). "Bayesian semiparametric clustering of functional predictors." URL:
  • Bouchard, G. (2007). "Efficient bounds for the Softmax Function and Applications to Approximate Inference in Hybrid models." In Neural Information Processing Systems Conference, Whistler, Canada, December 7-8.
  • Bougeard, S., Hanafi, M., and Qannari, E. (2008). "Continuum redundancy-PLS regression: A simple continuum approach." Computational Statistics and Data Analysis, 52: 3686–3696.
  • Brooks, R. and Stone, M. (1994). "Joint continuum regression with multiple predictands." Journal of the American Statistical Association, 89: 1374–1377.
  • Cardot, H., Crambes, C., Kneip, A., and Sarda, P. (2007). "Smoothing splines estimators in functional linear regression with errors-in-variables." Computational Statistics and Data Analysis, 51: 4832–4848.
  • Chiou, J.-M., Mueller, H.-G., and Wang, J.-L. (2004). "Functional response models." Statistica Sinica, 14: 659–677.
  • Choudrey, W., Penny, W., and Roberts, S. (2000). "An ensemble learning approach to independent component analysis." In Proceedings IEEE Workshop on Neural Networks for Signal Processing, Sydney, Australia, December 2000, 435–444. IEEE Press.
  • Choudrey, W. and Roberts, S. (2001). "Flexible Bayesian Independent Component Analysis for Blind Source Separation." In Proceedings of ICA-2001, San Diego, December 2001.
  • Davidian, M., Lin, X., and Wang, J.-L. (2004). "Emerging Issues in Longitudinal and Functional Data Analysis. Introduction." Statistica Sinica, 14: 613–614.
  • De la Cruz, R. (2008). "Bayesian non-linear regression models with skew-elliptical errors: Applications to the classification of longitudinal profiles." Computational Statistics and Data Analysis, 53: 436–449.
  • Dunson, D., Herring, A., and Siega-Riz, A. (2008). "Bayesian inference on changes in response densities over predictor clusters." Journal of the American Statistical Association, 103: 1508–1517.
  • Ferraty, F. and Vieu, P. (2006). Nonparametric Functional Data Analysis. New York: Springer.
  • Frank, I. and Friedman, H. (1993). "A Statistical View of Some Chemometrics Regression Tools." Technometrics, 35: 109–148. (with discussion).
  • Geweke, J. (1996). "Bayesian reduced rank regression in econometrics." Journal of Econometrics, 75: 121–146.
  • Hastie, T., Buja, A., and Tibshirani, R. (1995). "Penalized discriminant analysis." Annals of Statistics, 13: 435–475.
  • Hastie, T. and Little, F. (1987). "Principal Profiles." URL:\symbol126 hastie/Papers/PrincipalProfiles.pdf.
  • James (2002). "Generalized linear models with functional predictors." Journal of the Royal Statistal Society B, 64: 411–432.
  • James, G. and Hastie, T. (2001). "Functional linear discriminant analysis for irregularly sampled curves." Journal of the Royal Statistal Society B, 63: 533–550.
  • Karhunen, J. (2007). "Extending ICA for finding jointly dependent components for two related data sets." Neurocomputing, 70: 2969–2979.
  • Kaufman, C. and Sain, S. (2010). "Bayesian Functional ANOVA Modeling Using Gaussian Process Prior Distributions." Bayesian Analysis, 5: 123–150.
  • Klami, A. and Kaski, S. (2008). "Probabilistic approach to detecting dependencies between data sets." Neurocomputing, 72: 39–46.
  • Lopes, H. and West, M. (2004). "Bayesian Model Assessment in Factor Analysis." Statistica Sinica, 14: 41–67.
  • MacLehose, R. and Dunson, D. (2009). "Nonparametric Bayes kernel-based priors for functional data analysis." Statistica Sinica, 19: 611–629.
  • Manteiga, W. and Vieu, P. (2007). "Statistics for Functional Data." Computational Statistics and Data Analysis, 51: 4788–4792.
  • Mardia, K., Kent, J., and Bibby, J. (1979). Multivariate Analysis. London: Academic Press.
  • Merola, M. and Abraham, B. (2001). "Dimensionality reduction approach to multivariate prediction." Canadian Journal of Statistics, 29: 191–200.
  • Minka, T. (2001). "Automatic choice of dimensionality for PCA." In Leen, T., Dieterich, T., and Tresp, V. (eds.), Advances in Neural Information Processing Systems, volume 13, 598–604. MIT Press.
  • Morris, J., Brown, P., Baggerly, K., and Coombes, K. (2006). "Analysis of Mass Spectrometry Data Using Bayesian Wavelet-Based Functional Mixed Models." In Do, K., Mueller, P., and Vannucci, M. (eds.), Bayesian Inference for Gene Expression and Proteomics, 269–288. New York: Cambridge University Press.
  • Petrone, S., Guindani, M., and Gelfand, A. (2009). "Hybrid Dirichlet mixture models for functional data." Journal of the Royal Statistal Society B, 71: 755–782.
  • Preda, C. (2007). "Regression models for functional data by reproducing kernel Hilbert space methods." Journal of Statistical Planning and Inference, 137: 829–840.
  • Ramsay, J. and Silverman, B. (2002). Applied Functional Data Analysis: Methods and Case Studies. New York: Springer.
  • –- (2005). Functional Data Analysis. New York: Springer, 2nd edition.
  • Ray, S. and Mallick, B. (2006). "Functional clustering by Bayesian wavelet methods." Journal of the Royal Statistal Society B, 68: 305–332.
  • Reinsel, G. and Velu, R. (1998). Multivariate reduced-rank regression. New York: Springer.
  • –- (2003). "Reduced-rank growth curve models." Journal of Statistical Planning and Inference, 114: 107–129.
  • Rish, I., Grabarnik, G., Cecchi, G., Pereira, F., and Gordon, G. J. (2008). "Closed-form supervised dimensionality reduction with generalized linear models." In ACM Int. Conf. Proc. Series, volume 307, 832–839. New York: ACM.
  • Rodriguez, A., Dunson, D., and Gelfand, A. (2009). "Bayesian non-parametric functional data analysis through density estimation." Biometrika, 96: 149–162.
  • Schmidli, H. (1995). Reduced Rank Regression. Heidelberg: Physica-Verlag.
  • Smidl, V. and Quinn, A. (2007). "On Bayesian principal component analysis." Computational Statistics and Data Analysis, 51: 4101–4123.
  • Srivastava, M. (2007). "Reduced Rank Discrimination." Scandinavian Journal of Statistics, 24: 115–124.
  • Sundberg, R. (2002). "Continuum Regression." In Encyclopedia of Statistical Sciences. New York: Wiley.
  • Thompson, W. and Rosen, O. (2008). "A Bayesian model for sparse functional data." Biometrics, 64: 54–63.
  • Tipping, M. and Bishop, C. (1999). "Probabilistic principal component analysis." Journal of the Royal Statistal Society B, 21: 611–622.
  • Valderrama, M. (2007). "An overview to modelling functional data." Computational Statistics, 22: 331–334.
  • van der Linde, A. (2008). "Variational Bayesian functional PCA." Computational Statistics and Data Analysis, 53: 517–533.
  • –- (2009). "Bayesian functional principal components analysis for binary and count data." Advances in Statistical Analysis, 93: 307–333.
  • –- (2010). "Reduced rank regression in Bayesian FDA." Technical report. Department of Mathematics, Institute of Statistics, University of Bremen, Germany.
  • von Storch, H. and Zwiers, F. (1999). Statistical Analysis in Climate Research. Cambridge: Cambridge University Press.
  • Wang, C. (2007). "Variational Bayesian approach to canonical correlation analysis." IEEE Transactions on Neural Networks, 18: 905–910.
  • West, M. (2003). "Bayesian Factor Regression Models in the Large p, Small n Paradigm." In et al., J. B. (ed.), Bayesian Statistics 7, 733–742. Oxford University Press.
  • Yee, T. and Hastie, T. (2003). "Reduced-rank Vector Generalized Linear Models." Statistical Modelling, 3: 15–41.
  • Yu, S., Yu, K., Tresp, V., Kriegel, H.-P., and Wu, M. (2006). "Supervised probabilistic principal component analysis." In Proceedings of the 12th ACM SIGKDD international conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, 464–473. New York: ACM.
  • Zhang, D., Lin, X., and Sowers, M. (2007). "Two-Stage Functional Mixed Models for Evaluating the Effect of Longitudinal Covariate Profiles on Scalar Outcome." Biometrics, 63: 351–362.