The Annals of Statistics

Consistency of AIC and BIC in estimating the number of significant components in high-dimensional principal component analysis

Zhidong Bai, Kwok Pui Choi, and Yasunori Fujikoshi

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

In this paper, we study the problem of estimating the number of significant components in principal component analysis (PCA), which corresponds to the number of dominant eigenvalues of the covariance matrix of $p$ variables. Our purpose is to examine the consistency of the estimation criteria AIC and BIC based on the model selection criteria by Akaike [In 2nd International Symposium on Information Theory (1973) 267–281, Akadémia Kiado] and Schwarz [Estimating the dimension of a model 6 (1978) 461–464] under a high-dimensional asymptotic framework. Using random matrix theory techniques, we derive sufficient conditions for the criterion to be strongly consistent for the case when the dominant population eigenvalues are bounded, and when the dominant eigenvalues tend to infinity. Moreover, the asymptotic results are obtained without normality assumption on the population distribution. Simulation studies are also conducted, and results show that the sufficient conditions in our theorems are essential.

Article information

Source
Ann. Statist., Volume 46, Number 3 (2018), 1050-1076.

Dates
Received: October 2015
Revised: January 2017
First available in Project Euclid: 3 May 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1525313075

Digital Object Identifier
doi:10.1214/17-AOS1577

Mathematical Reviews number (MathSciNet)
MR3797996

Zentralblatt MATH identifier
1395.62119

Subjects
Primary: 62H12: Estimation
Secondary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]

Keywords
AIC BIC consistency dimensionality high-dimensional framework number of significant components principal component analysis random matrix theory signal processing spiked model

Citation

Bai, Zhidong; Choi, Kwok Pui; Fujikoshi, Yasunori. Consistency of AIC and BIC in estimating the number of significant components in high-dimensional principal component analysis. Ann. Statist. 46 (2018), no. 3, 1050--1076. doi:10.1214/17-AOS1577. https://projecteuclid.org/euclid.aos/1525313075


Export citation

References

  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd International Symposium on Information Theory (B. N. Petrov and F. Csáki, eds.) 267–281, Budapest: Akadémia Kiado.
  • Bai, Z. D., Miao, B. Q. and Rao, C. R. (1990). Estimation of direction of arrival of signals: Asymptotic results. In Advances in Spectrum Analysis and Array Processing (S. Haykins, ed.) 2 327–347. Prentice Hall, Englewood, Cliffs, NJ.
  • Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
  • Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167–177.
  • Bai, Z. D. and Yin, Y. Q. (1993). Limit of the smallest eigenvalue of a large-dimensional sample covariance matrix. Ann. Probab. 21 1275–1294.
  • Ferré, L. (1995). Selection of components in principal component analysis: A comparison of methods. Comput. Statist. Data Anal. 19 669–682.
  • Fujikoshi, Y. and Sakurai, T. (2016a). Some properties of estimation criteria for dimensionality in principal component anlysis. Amer. J. Math. Management Sci. 35 133–142.
  • Fujikoshi, Y. and Sakurai, T. (2016b). High-dimensional consistency of rank estimation criteria in multivariate linear model. J. Multivariate Anal. 149 199–212.
  • Fujikoshi, Y., Sakurai, T. and Yanagihara, H. (2014). Consistency of high-dimensional AIC-type and $C_{p}$-type criteria in multivariate linear regression. J. Multivariate Anal. 123 184–200.
  • Fujikoshi, Y., Ulyanov, V. V. and Shimizu, R. (2010). Multivariate Statistics: High-Dimensional and Large-Sample Approximations. Wiley, Hoboken, NJ.
  • Fujikoshi, Y., Yamada, T., Watanabe, D. and Sugiyama, T. (2007). Asymptotic distribution of the LR statistic for equality of the smallest eigenvalues in high-dimensional principal component analysis. J. Multivariate Anal. 98 2002–2008.
  • Gunderson, B. K. and Muirhead, R. J. (1997). On estimating the dimensionality in canonical correlation analysis. J. Multivariate Anal. 62 121–136.
  • Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682–693.
  • Jolliffe, I. T. (2002). Principal Component Analysis, 2nd ed. Springer, New York.
  • Jolliffe, I. T., Trendafilov, N. T. and Uddin, M. (2003). A modified principal component technique based on the LASSO. J. Comput. Graph. Statist. 12 531–547.
  • Kim, Y., Kwon, S. and Choi, H. (2012). Consistent model selection criteria on high dimensions. J. Mach. Learn. Res. 13 1037–1057.
  • Nishii, R. (1984). Asymptotic properties of criteria for selection of variables in multiple regression. Ann. Statist. 12 758–765.
  • Nishii, R., Bai, Z. D. and Krishnaiah, P. R. (1988). Strong consistency of the information criterion for model selection in multivariate analysis. Hiroshima Math. J. 18 451–462.
  • Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617–1642.
  • Rao, C. R. and Rao, M. B. (1998). Matrix Algebra and Its Applications to Statistics and Econometrics. World Scientific, River Edge, NJ.
  • Schott, J. R. (2006). A high-dimensional test for the equality of the smallest eigenvalues of a covariance matrix. J. Multivariate Anal. 97 827–843.
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7 221–264.
  • Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63 117–126.
  • Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
  • Yanagihara, H., Wakaki, H. and Fujikoshi, Y. (2015). A consistency property of the AIC for multivariate linear models when the dimension and the sample size are large. Electron. J. Stat. 9 869–897.
  • Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92 937–950.
  • Zhao, L. C., Krishnaiah, P. R. and Bai, Z. D. (1986). On detection of the number of signals in presence of white noise. J. Multivariate Anal. 20 1–25.
  • Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265–286.