## The Annals of Statistics

### Consistency of AIC and BIC in estimating the number of significant components in high-dimensional principal component analysis

#### Abstract

In this paper, we study the problem of estimating the number of significant components in principal component analysis (PCA), which corresponds to the number of dominant eigenvalues of the covariance matrix of $p$ variables. Our purpose is to examine the consistency of the estimation criteria AIC and BIC based on the model selection criteria by Akaike [In 2nd International Symposium on Information Theory (1973) 267–281, Akadémia Kiado] and Schwarz [Estimating the dimension of a model 6 (1978) 461–464] under a high-dimensional asymptotic framework. Using random matrix theory techniques, we derive sufficient conditions for the criterion to be strongly consistent for the case when the dominant population eigenvalues are bounded, and when the dominant eigenvalues tend to infinity. Moreover, the asymptotic results are obtained without normality assumption on the population distribution. Simulation studies are also conducted, and results show that the sufficient conditions in our theorems are essential.

#### Article information

Source
Ann. Statist., Volume 46, Number 3 (2018), 1050-1076.

Dates
Revised: January 2017
First available in Project Euclid: 3 May 2018

https://projecteuclid.org/euclid.aos/1525313075

Digital Object Identifier
doi:10.1214/17-AOS1577

Mathematical Reviews number (MathSciNet)
MR3797996

Zentralblatt MATH identifier
1395.62119

Subjects
Primary: 62H12: Estimation

#### Citation

Bai, Zhidong; Choi, Kwok Pui; Fujikoshi, Yasunori. Consistency of AIC and BIC in estimating the number of significant components in high-dimensional principal component analysis. Ann. Statist. 46 (2018), no. 3, 1050--1076. doi:10.1214/17-AOS1577. https://projecteuclid.org/euclid.aos/1525313075

#### References

• Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd International Symposium on Information Theory (B. N. Petrov and F. Csáki, eds.) 267–281, Budapest: Akadémia Kiado.
• Bai, Z. D., Miao, B. Q. and Rao, C. R. (1990). Estimation of direction of arrival of signals: Asymptotic results. In Advances in Spectrum Analysis and Array Processing (S. Haykins, ed.) 2 327–347. Prentice Hall, Englewood, Cliffs, NJ.
• Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
• Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167–177.
• Bai, Z. D. and Yin, Y. Q. (1993). Limit of the smallest eigenvalue of a large-dimensional sample covariance matrix. Ann. Probab. 21 1275–1294.
• Ferré, L. (1995). Selection of components in principal component analysis: A comparison of methods. Comput. Statist. Data Anal. 19 669–682.
• Fujikoshi, Y. and Sakurai, T. (2016a). Some properties of estimation criteria for dimensionality in principal component anlysis. Amer. J. Math. Management Sci. 35 133–142.
• Fujikoshi, Y. and Sakurai, T. (2016b). High-dimensional consistency of rank estimation criteria in multivariate linear model. J. Multivariate Anal. 149 199–212.
• Fujikoshi, Y., Sakurai, T. and Yanagihara, H. (2014). Consistency of high-dimensional AIC-type and $C_{p}$-type criteria in multivariate linear regression. J. Multivariate Anal. 123 184–200.
• Fujikoshi, Y., Ulyanov, V. V. and Shimizu, R. (2010). Multivariate Statistics: High-Dimensional and Large-Sample Approximations. Wiley, Hoboken, NJ.
• Fujikoshi, Y., Yamada, T., Watanabe, D. and Sugiyama, T. (2007). Asymptotic distribution of the LR statistic for equality of the smallest eigenvalues in high-dimensional principal component analysis. J. Multivariate Anal. 98 2002–2008.
• Gunderson, B. K. and Muirhead, R. J. (1997). On estimating the dimensionality in canonical correlation analysis. J. Multivariate Anal. 62 121–136.
• Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
• Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682–693.
• Jolliffe, I. T. (2002). Principal Component Analysis, 2nd ed. Springer, New York.
• Jolliffe, I. T., Trendafilov, N. T. and Uddin, M. (2003). A modified principal component technique based on the LASSO. J. Comput. Graph. Statist. 12 531–547.
• Kim, Y., Kwon, S. and Choi, H. (2012). Consistent model selection criteria on high dimensions. J. Mach. Learn. Res. 13 1037–1057.
• Nishii, R. (1984). Asymptotic properties of criteria for selection of variables in multiple regression. Ann. Statist. 12 758–765.
• Nishii, R., Bai, Z. D. and Krishnaiah, P. R. (1988). Strong consistency of the information criterion for model selection in multivariate analysis. Hiroshima Math. J. 18 451–462.
• Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617–1642.
• Rao, C. R. and Rao, M. B. (1998). Matrix Algebra and Its Applications to Statistics and Econometrics. World Scientific, River Edge, NJ.
• Schott, J. R. (2006). A high-dimensional test for the equality of the smallest eigenvalues of a covariance matrix. J. Multivariate Anal. 97 827–843.
• Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
• Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7 221–264.
• Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63 117–126.
• Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
• Yanagihara, H., Wakaki, H. and Fujikoshi, Y. (2015). A consistency property of the AIC for multivariate linear models when the dimension and the sample size are large. Electron. J. Stat. 9 869–897.
• Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92 937–950.
• Zhao, L. C., Krishnaiah, P. R. and Bai, Z. D. (1986). On detection of the number of signals in presence of white noise. J. Multivariate Anal. 20 1–25.
• Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265–286.