Annals of Statistics

Efficient estimation of linear functionals of principal components

Vladimir Koltchinskii, Matthias Löffler, and Richard Nickl

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We study principal component analysis (PCA) for mean zero i.i.d. Gaussian observations $X_{1},\dots,X_{n}$ in a separable Hilbert space $\mathbb{H}$ with unknown covariance operator $\Sigma $. The complexity of the problem is characterized by its effective rank $\mathbf{r}(\Sigma):=\frac{\operatorname{tr}(\Sigma)}{\|\Sigma \|}$, where $\mathrm{tr}(\Sigma)$ denotes the trace of $\Sigma $ and $\|\Sigma\|$ denotes its operator norm. We develop a method of bias reduction in the problem of estimation of linear functionals of eigenvectors of $\Sigma $. Under the assumption that $\mathbf{r}(\Sigma)=o(n)$, we establish the asymptotic normality and asymptotic properties of the risk of the resulting estimators and prove matching minimax lower bounds, showing their semiparametric optimality.

Article information

Source
Ann. Statist., Volume 48, Number 1 (2020), 464-490.

Dates
Received: September 2017
Revised: January 2019
First available in Project Euclid: 17 February 2020

Permanent link to this document
https://projecteuclid.org/euclid.aos/1581930143

Digital Object Identifier
doi:10.1214/19-AOS1816

Mathematical Reviews number (MathSciNet)
MR4065170

Zentralblatt MATH identifier
07196547

Subjects
Primary: 62H25: Factor analysis and principal components; correspondence analysis
Secondary: 62E17: Approximations to distributions (nonasymptotic)

Keywords
Principal component analysis spectral projections asymptotic normality semiparametric efficiency

Citation

Koltchinskii, Vladimir; Löffler, Matthias; Nickl, Richard. Efficient estimation of linear functionals of principal components. Ann. Statist. 48 (2020), no. 1, 464--490. doi:10.1214/19-AOS1816. https://projecteuclid.org/euclid.aos/1581930143


Export citation

References

  • [1] Anderson, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Stat. 34 122–148.
  • [2] Benaych-Georges, F. and Nadakuditi, R. R. (2011). The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math. 227 494–521.
  • [3] Berthet, Q. and Rigollet, P. (2013). Optimal detection of sparse principal components in high dimension. Ann. Statist. 41 1780–1815.
  • [4] Blanchard, G., Bousquet, O. and Zwald, L. (2007). Statistical properties of kernel principal component analysis. Mach. Learn. 66 259–294.
  • [5] Bloemendal, A., Knowles, A., Yau, H.-T. and Yin, J. (2016). On the principal components of sample covariance matrices. Probab. Theory Related Fields 164 459–552.
  • [6] Cai, T. T. and Guo, Z. (2017). Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity. Ann. Statist. 45 615–646.
  • [7] Cai, T. T., Ma, Z. and Wu, Y. (2013). Sparse PCA: Optimal rates and adaptive estimation. Ann. Statist. 41 3074–3110.
  • [8] Dauxois, J., Pousse, A. and Romain, Y. (1982). Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference. J. Multivariate Anal. 12 136–154.
  • [9] Eaton, M. L. (1983). Multivariate Statistics: A Vector Space Approach. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. Wiley, New York.
  • [10] Fan, J., Rigollet, P. and Wang, W. (2015). Estimation of functionals of sparse covariance matrices. Ann. Statist. 43 2706–2737.
  • [11] Gao, C. and Zhou, H. H. (2015). Rate-optimal posterior contraction for sparse PCA. Ann. Statist. 43 785–818.
  • [12] Gao, C. and Zhou, H. H. (2016). Bernstein–von Mises theorems for functionals of the covariance matrix. Electron. J. Stat. 10 1751–1806.
  • [13] Gill, R. D. and Levit, B. Y. (1995). Applications of the Van Trees inequality: A Bayesian Cramér–Rao bound. Bernoulli 1 59–79.
  • [14] Giné, E. and Nickl, R. (2016). Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge Univ. Press, New York.
  • [15] Janková, J. and van de Geer, S. (2018). Semiparametric efficiency bounds for high-dimensional models. Ann. Statist. 46 2336–2359.
  • [16] Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. J. Mach. Learn. Res. 15 2869–2909.
  • [17] Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • [18] Koltchinskii, V. (2017). Asymptotically efficient estimation of smooth functionals of covariance operators. Available at https://arxiv.org/abs/1710.09072.
  • [19] Koltchinskii, V., Löffler, M. and Nickl, R. (2019). Supplement to “Efficient estimation of linear functionals of principal components.” https://doi.org/10.1214/19-AOS1816SUPP.
  • [20] Koltchinskii, V. and Lounici, K. (2016). Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance. Ann. Inst. Henri Poincaré Probab. Stat. 52 1976–2013.
  • [21] Koltchinskii, V. and Lounici, K. (2017). Concentration inequalities and moment bounds for sample covariance operators. Bernoulli 23 110–133.
  • [22] Koltchinskii, V. and Lounici, K. (2017). Normal approximation and concentration of spectral projectors of sample covariance. Ann. Statist. 45 121–157.
  • [23] Koltchinskii, V. and Lounici, K. (2017). New asymptotic results in principal component analysis. Sankhya A 79 254–297.
  • [24] Lila, E., Aston, J. A. D. and Sangalli, L. M. (2016). Smooth principal component analysis over two-dimensional manifolds with an application to neuroimaging. Ann. Appl. Stat. 10 1854–1879.
  • [25] Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791–2817.
  • [26] Naumov, A., Spokoiny, V. and Ulyanov, V. (2018). Confidence sets for spectral projectors of covariance matrices. Dokl. Math. 98 511–514.
  • [27] Ning, Y. and Liu, H. (2017). A general theory of hypothesis tests and confidence regions for sparse high dimensional models. Ann. Statist. 45 158–195.
  • [28] Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617–1642.
  • [29] Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer Series in Statistics. Springer, New York.
  • [30] Reiss, M. and Wahl, M. (2016). Non-asymptotic upper bounds for the reconstruction error of PCA. Available at arXiv:1609.03779.
  • [31] Ren, Z., Sun, T., Zhang, C.-H. and Zhou, H. H. (2015). Asymptotic normality and optimalities in estimation of large Gaussian graphical models. Ann. Statist. 43 991–1026.
  • [32] van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Statist. 42 1166–1202.
  • [33] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [34] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York.
  • [35] Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge Univ. Press, Cambridge.
  • [36] Vu, V. Q. and Lei, J. (2013). Minimax sparse principal subspace estimation in high dimensions. Ann. Statist. 41 2905–2947.
  • [37] Wang, T., Berthet, Q. and Samworth, R. J. (2016). Statistical and computational trade-offs in estimation of sparse principal components. Ann. Statist. 44 1896–1930.
  • [38] Wang, W. and Fan, J. (2017). Asymptotics of empirical eigenstructure for high dimensional spiked covariance. Ann. Statist. 45 1342–1374.
  • [39] Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 217–242.

Supplemental materials

  • Supplement to “Efficient estimation of linear functionals of principal components”. Supplementary information.