The Annals of Statistics

Covariance matrix estimation for stationary time series

Han Xiao and Wei Biao Wu

Full-text: Open access


We obtain a sharp convergence rate for banded covariance matrix estimates of stationary processes. A precise order of magnitude is derived for spectral radius of sample covariance matrices. We also consider a thresholded covariance matrix estimator that can better characterize sparsity if the true covariance matrix is sparse. As our main tool, we implement Toeplitz [Math. Ann. 70 (1911) 351–376] idea and relate eigenvalues of covariance matrices to the spectral densities or Fourier transforms of the covariances. We develop a large deviation result for quadratic forms of stationary processes using m-dependence approximation, under the framework of causal representation and physical dependence measures.

Article information

Ann. Statist., Volume 40, Number 1 (2012), 466-493.

First available in Project Euclid: 16 April 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]
Secondary: 62H12: Estimation

Autocovariance matrix banding large deviation physical dependence measure short range dependence spectral density stationary process tapering thresholding Toeplitz matrix


Xiao, Han; Wu, Wei Biao. Covariance matrix estimation for stationary time series. Ann. Statist. 40 (2012), no. 1, 466--493. doi:10.1214/11-AOS967.

Export citation


  • Adenstedt, R. K. (1974). On large-sample estimation for the mean of a stationary random sequence. Ann. Statist. 2 1095–1107.
  • An, H. Z., Chen, Z. G. and Hannan, E. J. (1983). The maximum of the periodogram. J. Multivariate Anal. 13 383–400.
  • Andrews, D. W. K. (1984). Nonstrong mixing autoregressive processes. J. Appl. Probab. 21 930–934.
  • Bai, Z. and Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Springer, New York.
  • Bai, Z. D. and Yin, Y. Q. (1993). Limit of the smallest eigenvalue of a large-dimensional sample covariance matrix. Ann. Probab. 21 1275–1294.
  • Bentkus, R. and Rudzkis, R. (1976). Large deviations for estimates of the spectrum of a stationary Gaussian sequence. Litovsk. Mat. Sb. 16 63–77, 253.
  • Bercu, B., Gamboa, F. and Rouault, A. (1997). Large deviations for quadratic forms of stationary Gaussian processes. Stochastic Process. Appl. 71 75–90.
  • Bercu, B., Gamboa, F. and Lavielle, M. (2000). Sharp large deviations for Gaussian quadratic forms with applications. ESAIM Probab. Stat. 4 1–24 (electronic).
  • Bickel, P. J. and Levina, E. (2008a). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.
  • Bickel, P. J. and Levina, E. (2008b). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
  • Bryc, W. and Dembo, A. (1997). Large deviations for quadratic functionals of Gaussian processes. J. Theoret. Probab. 10 307–332. Dedicated to Murray Rosenblatt.
  • Bryc, W., Dembo, A. and Jiang, T. (2006). Spectral measure of large random Hankel, Markov and Toeplitz matrices. Ann. Probab. 34 1–38.
  • Bühlmann, P. and Künsch, H. R. (1999). Block length selection in the bootstrap for time series. Comput. Statist. Data Anal. 31 295–310.
  • Burkholder, D. L. (1988). Sharp inequalities for martingales and stochastic integrals. Colloque Paul Lévy sur les Processus Stochastiques (Palaiseau, 1987). Astérisque 157-158 75–94.
  • Cai, T. T., Zhang, C.-H. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118–2144.
  • Dembo, A. and Zeitouni, O. (1998). Large Deviations Techniques and Applications, 2nd ed. Applications of Mathematics (New York) 38. Springer, New York.
  • Djellout, H., Guillin, A. and Wu, L. (2006). Moderate deviations of empirical periodogram and non-linear functionals of moving average processes. Ann. Inst. Henri Poincaré Probab. Stat. 42 393–416.
  • El Karoui, N. (2005). Recent results about the largest eigenvalue of random covariance matrices and statistical application. Acta Phys. Polon. B 36 2681–2697.
  • El Karoui, N. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717–2756.
  • Freedman, D. A. (1975). On tail probabilities for martingales. Ann. Probab. 3 100–118.
  • Furrer, R. and Bengtsson, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivariate Anal. 98 227–255.
  • Gamboa, F., Rouault, A. and Zani, M. (1999). A functional large deviations principle for quadratic forms of Gaussian stationary processes. Statist. Probab. Lett. 43 299–308.
  • Geman, S. (1980). A limit theorem for the norm of random matrices. Ann. Probab. 8 252–261.
  • Grenander, U. and Szegö, G. (1958). Toeplitz Forms and Their Applications. Univ. California Press, Berkeley.
  • Haeusler, E. (1984). An exact rate of convergence in the functional central limit theorem for special martingale difference arrays. Z. Wahrsch. Verw. Gebiete 65 523–534.
  • Horn, R. A. and Johnson, C. R. (1990). Matrix Analysis. Cambridge Univ. Press, Cambridge. Corrected reprint of the 1985 original.
  • Johansson, K. (2000). Shape fluctuations and random matrices. Comm. Math. Phys. 209 437–476.
  • Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • Kakizawa, Y. (2007). Moderate deviations for quadratic forms in Gaussian stationary processes. J. Multivariate Anal. 98 992–1017.
  • Kolmogoroff, A. (1941). Interpolation und Extrapolation von stationären zufälligen Folgen. Bull. Acad. Sci. URSS Sér. Math. [Izvestia Akad. Nauk SSSR] 5 3–14.
  • Lin, Z. and Liu, W. (2009). On maxima of periodograms of stationary processes. Ann. Statist. 37 2676–2695.
  • Liu, W. and Shao, Q.-M. (2010). Cramér-type moderate deviation for the maximum of the periodogram with application to simultaneous tests in gene expression time series. Ann. Statist. 38 1913–1935.
  • Liu, W. and Wu, W. B. (2010). Asymptotics of spectral density estimates. Econometric Theory 26 1218–1245.
  • Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. (N.S.) 72 507–536.
  • McMurry, T. L. and Politis, D. N. (2010). Banded and tapered estimates for autocovariance matrices and the linear process bootstrap. J. Time Series Anal. 31 471–482.
  • Nagaev, S. V. (1979). Large deviations of sums of independent random variables. Ann. Probab. 7 745–789.
  • Peligrad, M. and Wu, W. B. (2010). Central limit theorem for Fourier transforms of stationary processes. Ann. Probab. 38 2009–2022.
  • Politis, D. N. (2003). Adaptive bandwidth choice. J. Nonparametr. Stat. 15 517–533.
  • Politis, D. N., Romano, J. P. and Wolf, M. (1999). Subsampling. Springer, New York.
  • Rio, E. (2009). Moment inequalities for sums of dependent random variables under projective conditions. J. Theoret. Probab. 22 146–163.
  • Rudzkis, R. (1978). Large deviations for estimates of the spectrum of a stationary sequence. Litovsk. Mat. Sb. 18 81–98, 217.
  • Saulis, L. and Statulevičius, V. A. (1991). Limit Theorems for Large Deviations. Mathematics and Its Applications (Soviet Series) 73. Kluwer, Dordrecht. Translated and revised from the 1989 Russian original.
  • Shao, X. and Wu, W. B. (2007). Asymptotic spectral theory for nonlinear time series. Ann. Statist. 35 1773–1801.
  • Solo, V. (2010). On random matrix theory for stationary processes. In IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) 3758–3761. IEEE, Piscataway, NJ.
  • Toeplitz, O. (1911). Zur Theorie der quadratischen und bilinearen Formen von unendlichvielen Veränderlichen. Math. Ann. 70 351–376.
  • Tong, H. (1990). Nonlinear Time Series. Oxford Statistical Science Series 6. Oxford Univ. Press, New York.
  • Tracy, C. A. and Widom, H. (1994). Level-spacing distributions and the Airy kernel. Comm. Math. Phys. 159 151–174.
  • Turkman, K. F. and Walker, A. M. (1984). On the asymptotic distributions of maxima of trigonometric polynomials with random coefficients. Adv. in Appl. Probab. 16 819–842.
  • Turkman, K. F. and Walker, A. M. (1990). A stability result for the periodogram. Ann. Probab. 18 1765–1783.
  • Wiener, N. (1949). Extrapolation, Interpolation, and Smoothing of Stationary Time Series. With Engineering Applications. MIT Press, Cambridge, MA.
  • Woodroofe, M. B. and Van Ness, J. W. (1967). The maximum deviation of sample spectral densities. Ann. Math. Statist. 38 1558–1569.
  • Wu, W. B. (2005). Nonlinear system theory: Another look at dependence. Proc. Natl. Acad. Sci. USA 102 14150–14154 (electronic).
  • Wu, W. B. and Pourahmadi, M. (2009). Banding sample autocovariance matrices of stationary processes. Statist. Sinica 19 1755–1768.
  • Xiao, H. and Wu, W. B. (2011). Asymptotic inference of autocovariances of stationary processes. Available at arXiv:1105.3423.
  • Xiao, H. and Wu, W. B. (2012). Supplement to “Covariance matrix estimation for stationary time series.” DOI:10.1214/11-AOS967SUPP.
  • Yin, Y. Q., Bai, Z. D. and Krishnaiah, P. R. (1988). On the limit of the largest eigenvalue of the large-dimensional sample covariance matrix. Probab. Theory Related Fields 78 509–521.
  • Zani, M. (2002). Large deviations for quadratic forms of locally stationary processes. J. Multivariate Anal. 81 205–228.
  • Zygmund, A. (2002). Trigonometric Series. Vols I, II, 3rd ed. Cambridge Univ. Press, Cambridge.

Supplemental materials