Electronic Journal of Statistics

Explicit connections between longitudinal data analysis and kernel machines

N.D. Pearce and M.P. Wand

Full-text: Open access


Two areas of research – longitudinal data analysis and kernel machines – have large, but mostly distinct, literatures. This article shows explicitly that both fields have much in common with each other. In particular, many popular longitudinal data fitting procedures are special types of kernel machines. These connections have the potential to provide fruitful cross-fertilization between longitudinal data analytic and kernel machine methodology.

Article information

Electron. J. Statist. Volume 3 (2009), 797-823.

First available in Project Euclid: 12 August 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Best linear unbiased prediction classification generalized linear mixed models machine learning linear mixed models reproducing kernel Hilbert spaces penalized likelihood support vector machines


Pearce, N.D.; Wand, M.P. Explicit connections between longitudinal data analysis and kernel machines. Electron. J. Statist. 3 (2009), 797--823. doi:10.1214/09-EJS428. https://projecteuclid.org/euclid.ejs/1250085687

Export citation


  • [1] Aronszajn, N. (1950). Theory of reproducing kernels., Transactions of the American Mathematical Society, 68, 337–404.
  • [2] Bachrach, L.K., Hastie, T., Wang, M.-C., Narasimhan, B. and Marcus, R. (1999). Bone mineral acquisition in healthy Asian, Hispanic, Black and Caucasian youth. A longitudinal study., Journal of Clinical Endocrinology and Metabolism, 84, 4702–12.
  • [3] Breslow, N.E. and Clayton, D.G. (1993). Approximate inference in generalized linear mixed models., Journal of the American Statistical Association, 88, 9–25.
  • [4] Cristianini, N. and Shawe-Taylor, J. (2000)., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge: Cambridge University Press.
  • [5] Cressie, N. (1993)., Statistics for Spatial Data. New York: John Wiley & Sons.
  • [6] Diggle, P.J., Heagerty, P., Liang, K.-L. and Zeger, S. (2002)., Analysis of Longitudinal Data (Second Edition). Oxford: Oxford University Press.
  • [7] Evgeniou, T., Pontil, M. and Poggio, T. (2000). Regularization networks and support vector machines., Advances in Computational Mathematics, 13, 1–50.
  • [8] Fitzmaurice, G., Davidian, M., Verbeke, G. and Molenberghs, G. (Eds.) (2008)., Longitudinal Data Analysis: A Handbook of Modern Statistical Methods. Boca Raton, Florida: Chapman & Hall/CRC.
  • [9] Fitzmaurice, G.M., Laird, N.M. and Ware, J.H. (2004)., Applied Longitudinal Analysis. New York: John Wiley & Sons.
  • [10] Gianola, D., Fernando, R.L. and Stella, A. (2006). Genomic-assisted prediction of genetic value with semiparametric procedures., Genetics, 173, 1761–1776.
  • [11] Green, P.J. and Silverman, B.W. (1994)., Nonparametric Regression and Generalized Linear Models. London: Chapman & Hall.
  • [12] Harville, D.A. (1977). Maximum likelihood approaches to variance component estimation and to related problems., Journal of the American Statistical Association, 72, 320–338.
  • [13] Hastie, T.J. and Tibshirani, R.J. (1990)., Generalized Additive Models. London: Chapman & Hall.
  • [14] Huber P. (1981)., Robust Statistics. Chichester: John Wiley & Sons.
  • [15] James, G.M. and Hastie, T.J. (2001). Functional linear discriminant analysis for irregularly sampled curves., Journal of the Royal Statistical Society, Series B, 63, 533–550.
  • [16] Kimeldorf, G. and Wahba, G. (1971). Some results on Tchebycheffian spline functions., Journal of Mathematical Analysis and Its Application, 33, 82–95.
  • [17] Laird, N.M. and Ware, J.H. (1982). Random-effects models for longitudinal data., Biometrics, 38, 963–974.
  • [18] Liu, D., Lin, X. and Ghosh, D. (2007). Semiparametric regression of multidimensional genetic pathway data: least-squares kernel machines and linear mixed models., Biometrics, 63, 1079–1088.
  • [19] McCulloch, C.E., Searle, S.R. and Neuhaus, J.M. (2008)., Generalized, Linear, and Mixed Models, Second Edition. New York: John Wiley & Sons.
  • [20] Moguerza, J.M. and Muñoz, A. (2006). Support vector machines with applications (with discussion)., Statistical Science, 21, 322–362.
  • [21] Müller, H.-G. (2005). Functional modelling and classification of longitudinal data., Scandinavian Journal of Statistics, 32, 223–240.
  • [22] Ormerod, J.T. and Wand, M.P. (2006). LowRankQP 1.0. R package., http://cran.r-project.org
  • [23] Ormerod, J.T., Wand, M.P. and Koch, I. (2008). Penalised spline support vector classifiers: computational issues., Computational Statistics, 23, 623–641.
  • [24] Pearce, N.D. and Wand, M.P. (2006). Penalized splines and reproducing kernel methods., The American Statistician, 60, 233–240.
  • [25] Rasmussen, C.E. and Williams, K.I. (2006)., Gaussian Processes for Machine Learning, The MIT Press.
  • [26] Robinson, G.K. (1991). That BLUP is a good thing: the estimation of random effects., Statistical Science, 6, 15–51.
  • [27] Rudin, W. (1991)., Functional Analysis, Second Edition. New York: McGraw Hill.
  • [28] Ruppert, D., Wand, M. P. and Carroll, R.J. (2003)., Semiparametric Regression. New York: Cambridge University Press.
  • [29] Schölkopf, B. and Smola, A.J. (2002)., Learning with Kernels. Cambridge USA: MIT Press.
  • [30] Shen, X., Tseng, G.C., Zhang, X. and Wong, W. (2003). On, ψ-learning. Journal of the American Statistical Association, 98, 724–734.
  • [31] Simmons, G.F. (1983)., Introduction to Topology and Modern Analysis. Melbourne USA: Krieger.
  • [32] Stein, M.L. (1999)., Interpolation of Spatial Data: Some Theory for Kriging. New York: Springer.
  • [33] Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B. and Vandewall, J. (2002), Least squares support vector machines. Singapore: World Scientific Publishing Company.
  • [34] Tarantola, A. (2005)., Inverse Problem Theory. Philadelphia: Society of Industrial and Applied Mathematics.
  • [35] Verbeke, G. and Molenberghs, G. (2000)., Linear Mixed Models for Longitudinal Data. New York: Springer-Verlag.
  • [36] Wahba, G. (1990)., Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics.
  • [37] Wahba, G. (1999). Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. In, Advances in Kernel Methods : Support Vector Learning (eds. B. Scholkopf, C. Burges and A. Smola) pp. 69–88. Cambridge USA: MIT Press.
  • [38] Wand, M.P. and Ormerod, J.T. (2008). On O’Sullivan penalised splines and semiparametric regression., Australian and New Zealand Journal of Statistics, 50, 179–198.
  • [39] Wang, J. and Shen, X. (2007). Large margin semi-supervised learning., Journal of Machine Learning Research, 8, 1867–1891.
  • [40] Zhu, J. and Hastie, T. (2005). Kernel logistic regression and the import vector machine., Journal of Computational and Graphical Statistics, 14, 185–205.