The Annals of Statistics

Consistency of restricted maximum likelihood estimators of principal components

Debashis Paul and Jie Peng

Full-text: Open access


In this paper we consider two closely related problems: estimation of eigenvalues and eigenfunctions of the covariance kernel of functional data based on (possibly) irregular measurements, and the problem of estimating the eigenvalues and eigenvectors of the covariance matrix for high-dimensional Gaussian vectors. In [A geometric approach to maximum likelihood estimation of covariance kernel from sparse irregular longitudinal data (2007)], a restricted maximum likelihood (REML) approach has been developed to deal with the first problem. In this paper, we establish consistency and derive rate of convergence of the REML estimator for the functional data case, under appropriate smoothness conditions. Moreover, we prove that when the number of measurements per sample curve is bounded, under squared-error loss, the rate of convergence of the REML estimators of eigenfunctions is near-optimal. In the case of Gaussian vectors, asymptotic consistency and an efficient score representation of the estimators are obtained under the assumption that the effective dimension grows at a rate slower than the sample size. These results are derived through an explicit utilization of the intrinsic geometry of the parameter space, which is non-Euclidean. Moreover, the results derived in this paper suggest an asymptotic equivalence between the inference on functional data with dense measurements and that of the high-dimensional Gaussian vectors.

Article information

Ann. Statist., Volume 37, Number 3 (2009), 1229-1271.

First available in Project Euclid: 10 April 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G20: Asymptotic properties 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]
Secondary: 62H25: Factor analysis and principal components; correspondence analysis

Functional data analysis principal component analysis high-dimensional data Stiefel manifold intrinsic geometry consistency


Paul, Debashis; Peng, Jie. Consistency of restricted maximum likelihood estimators of principal components. Ann. Statist. 37 (2009), no. 3, 1229--1271. doi:10.1214/08-AOS608.

Export citation


  • [1] Antoniadis, A. and Sapatinas, T. (2007). Estimation and inference in functional mixed-effects models. Comput. Statist. Data Anal. 51 4793–4813.
  • [2] Bickel, P. J. and Levina, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.
  • [3] Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
  • [4] Burman, P. (1985). A data dependent approach to density estimation. Z. Wahrsch. Verw. Gebiete 69 609–628.
  • [5] Chen, A. and Bickel, P. J. (2006). Efficient independent component analysis. Ann. Statist. 34 2825–2855.
  • [6] de Boor, C. (1974). Bounding the error in spline interpolation. SIAM Rev. 16 531–544.
  • [7] de Boor, C. (1978). A Practical Guide to Splines. Applied Mathematical Sciences 27. Springer, New York.
  • [8] deVore, R. A. and Lorentz, G. G. (1993). Constructive Approximation. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 303. Springer, Berlin.
  • [9] El Karoui, N. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. To appear.
  • [10] Davidson, K. R. and Szarek, S. (2001). Local operator theory, random matrices and Banach spaces. In Handbook on the Geometry of Banach spaces (W. B. Johnson and J. Lendenstrauss, eds.) 1 317–366. North-Holland, Amsterdam.
  • [11] Edelman, A., Arias, T. A. and Smith, S. T. (1998). The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20 303–353.
  • [12] Fan, J., Fan, Y. and Lv, J. (2008). High-dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186–197.
  • [13] Ferraty, F. and Vieu, P. (2006). Nonparametric Functional Data Analysis: Theory and Practice. Springer, New York.
  • [14] Hall, P. and Hosseini-Nasab, M. (2006). On properties of functional principal components analysis. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 109–126.
  • [15] Hall, P., Müller, H.-G. and Wang, J.-L. (2006). Properties of principal component methods for functional and longitudinal data analysis. Ann. Statist. 34 1493–1517.
  • [16] Horn, R. A. and Johnson, C. R. (1994). Topics in Matrix Analysis. Cambridge Univ. Press, Cambridge.
  • [17] James, G. M., Hastie, T. J. and Sugar, C. A. (2000). Principal component models for sparse functional data. Biometrika 87 587–602.
  • [18] Kato, T. (1980). Perturbation Theory of Linear Operators, 2nd ed. Springer, Berlin.
  • [19] Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York.
  • [20] Oller, J. M. and Corcuera, J. M. (1995). Intrinsic analysis of statistical estimation. Ann. Statist. 23 1562–1581.
  • [21] Paul, D. (2005). Nonparametric estimation of principal components. Ph.D. dissertation, Stanford Univ.
  • [22] Paul, D. and Johnstone, I. M. (2007). Augmented sparse principal component analysis for high-dimensional data. Working paper. Available at
  • [23] Peng, J. and Paul, D. (2007). A geometric approach to maximum likelihood estimation of covariance kernel from sparse irregular longitudinal data. Technical report. arXiv:0710.5343v1. Avaialble at
  • [24] Ramsay, J. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • [25] Yang, A. and Barron, A. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564–1599.
  • [26] Yao, F., Müller, H.-G. and Wang, J.-L. (2005). Functional data analysis for sparse longitudinal data. J. Amer. Statist. Assoc. 100 577–590.