The Annals of Statistics

Covariance matrix estimation and linear process bootstrap for multivariate time series of possibly increasing dimension

Carsten Jentsch and Dimitris N. Politis

Full-text: Open access


Multivariate time series present many challenges, especially when they are high dimensional. The paper’s focus is twofold. First, we address the subject of consistently estimating the autocovariance sequence; this is a sequence of matrices that we conveniently stack into one huge matrix. We are then able to show consistency of an estimator based on the so-called flat-top tapers; most importantly, the consistency holds true even when the time series dimension is allowed to increase with the sample size. Second, we revisit the linear process bootstrap (LPB) procedure proposed by McMurry and Politis [ J. Time Series Anal. 31 (2010) 471–482] for univariate time series. Based on the aforementioned stacked autocovariance matrix estimator, we are able to define a version of the LPB that is valid for multivariate time series. Under rather general assumptions, we show that our multivariate linear process bootstrap (MLPB) has asymptotic validity for the sample mean in two important cases: (a) when the time series dimension is fixed and (b) when it is allowed to increase with sample size. As an aside, in case (a) we show that the MLPB works also for spectral density estimators which is a novel result even in the univariate case. We conclude with a simulation study that demonstrates the superiority of the MLPB in some important cases.

Article information

Ann. Statist., Volume 43, Number 3 (2015), 1117-1140.

Received: January 2014
Revised: September 2014
First available in Project Euclid: 15 May 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G09: Resampling methods
Secondary: 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]

Asymptotics bootstrap covariance matrix high-dimensional data multivariate time series sample mean spectral density


Jentsch, Carsten; Politis, Dimitris N. Covariance matrix estimation and linear process bootstrap for multivariate time series of possibly increasing dimension. Ann. Statist. 43 (2015), no. 3, 1117--1140. doi:10.1214/14-AOS1301.

Export citation


  • Brillinger, D. R. (1981). Time Series: Data Analysis and Theory, 2nd ed. Holden-Day, Oakland, CA.
  • Brockwell, P. J. and Davis, R. A. (1988). Simple consistent estimation of the coefficients of a linear filter. Stochastic Process. Appl. 28 47–59.
  • Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Methods, 2nd ed. Springer Series in Statistics. Springer, New York.
  • Bühlmann, P. (1997). Sieve bootstrap for time series. Bernoulli 3 123–148.
  • Bühlmann, P. (2002). Bootstraps for time series. Statist. Sci. 17 52–72.
  • Cai, T. T., Ren, Z. and Zhou, H. H. (2013). Optimal rates of convergence for estimating Toeplitz covariance matrices. Probab. Theory Related Fields 156 101–143.
  • Davidson, J. (1994). Stochastic Limit Theory: An Introduction for Econometricians. Oxford Univ. Press, New York.
  • Dedecker, J., Doukhan, P., Lang, G., León R., J. R., Louhichi, S. and Prieur, C. (2007). Weak Dependence: With Examples and Applications. Lecture Notes in Statistics 190. Springer, New York.
  • Doukhan, P. (1994). Mixing: Properties and Examples. Lecture Notes in Statistics 85. Springer, New York.
  • Gonçalves, S. and Kilian, L. (2007). Asymptotic and bootstrap inference for $\mathrm{AR}(\infty)$ processes with conditional heteroskedasticity. Econometric Rev. 26 609–641.
  • Hannan, E. J. (1970). Multiple Time Series. Wiley, New York.
  • Härdle, W., Horowitz, J. and Kreiss, J.-P. (2003). Bootstrap methods for time series. Int. Stat. Rev. 71 435–459.
  • Horn, R. A. and Johnson, C. R. (1990). Matrix Analysis. Cambridge Univ. Press, Cambridge. Corrected reprint of the 1985 original.
  • Jentsch, C. and Politis, D. N. (2013). Valid resampling of higher-order statistics using the linear process bootstrap and autoregressive sieve bootstrap. Comm. Statist. Theory Methods 42 1277–1293.
  • Jentsch, C. and Politis, D. N. (2015). Supplement to “Covariance matrix estimation and linear process bootstrap for multivariate time series of possibly increasing dimension.” DOI:10.1214/14-AOS1301SUPP.
  • Kreiss, J.-P. (1992). Bootstrap procedures for $\mathrm{AR}(\infty)$-processes. In Bootstrapping and Related Techniques (Trier, 1990). Lecture Notes in Econom. and Math. Systems 376 107–113. Springer, Berlin.
  • Kreiss, J.-P. (1999). Residual and wild bootstrap for infinite order autoregression. Unpublished manuscript.
  • Kreiss, J.-P. and Paparoditis, E. (2011). Bootstrap methods for dependent data: A review. J. Korean Statist. Soc. 40 357–378.
  • Kreiss, J.-P., Paparoditis, E. and Politis, D. N. (2011). On the range of validity of the autoregressive sieve bootstrap. Ann. Statist. 39 2103–2130.
  • Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. Ann. Statist. 17 1217–1241.
  • Lahiri, S. N. (2003). Resampling Methods for Dependent Data. Springer, New York.
  • Lewis, R. and Reinsel, G. C. (1985). Prediction of multivariate time series by autoregressive model fitting. J. Multivariate Anal. 16 393–411.
  • Liu, R. Y. and Singh, K. (1992). Moving blocks jackknife and bootstrap capture weak dependence. In Exploring the Limits of Bootstrap (East Lansing, MI, 1990) (R. LePage and L. Billard, eds.). Wiley Ser. Probab. Math. Statist. Probab. Math. Statist. 225–248. Wiley, New York.
  • McMurry, T. L. and Politis, D. N. (2010). Banded and tapered estimates for autocovariance matrices and the linear process bootstrap. J. Time Series Anal. 31 471–482. Corrigendum: J. Time Ser. Anal. 33 (2012).
  • Mitchell, H. and Brockwell, P. (1997). Estimation of the coefficients of a multivariate linear filter using the innovations algorithm. J. Time Series Anal. 18 157–179.
  • Paparoditis, E. (2002). Frequency domain bootstrap for time series. In Empirical Process Techniques for Dependent Data 365–381. Birkhäuser, Boston, MA.
  • Politis, D. N. (2001). On nonparametric function estimation with infinite-order flat-top kernels. In Probability and Statistical Models with Applications (Ch. Charalambides et al., eds.) 469–483. Chapman & Hall/CRC, Boca Raton.
  • Politis, D. N. (2003a). The impact of bootstrap methods on time series analysis: Silver anniversary of the bootstrap. Statist. Sci. 18 219–230.
  • Politis, D. N. (2003b). Adaptive bandwidth choice. J. Nonparametr. Stat. 15 517–533.
  • Politis, D. N. (2011). Higher-order accurate, positive semidefinite estimation of large-sample covariance and spectral density matrices. Econometric Theory 27 703–744.
  • Politis, D. N. and Romano, J. P. (1992). A general resampling scheme for triangular arrays of $\alpha$-mixing random variables with application to the problem of spectral density estimation. Ann. Statist. 20 1985–2007.
  • Politis, D. N. and Romano, J. P. (1994). Limit theorems for weakly dependent Hilbert space valued random variables with application to the stationary bootstrap. Statist. Sinica 4 461–476.
  • Politis, D. N. and Romano, J. P. (1995). Bias-corrected nonparametric spectral estimation. J. Time Series Anal. 16 67–103.
  • Rissanen, J. and Barbosa, L. (1969). Properties of infinite covariance matrices and stability of optimum predictors. Information Sci. 1 221–236.
  • Shao, X. (2010). The dependent wild bootstrap. J. Amer. Statist. Assoc. 105 218–235.
  • Wu, W. B. and Pourahmadi, M. (2009). Banding sample autocovariance matrices of stationary processes. Statist. Sinica 19 1755–1768.

Supplemental materials

  • Additional proofs, simulations and a real data example. In the supplementary material we provide proofs, additional supporting simulations and an application of the MLPB to German stock index data. The supplementary material to this paper is also available online at