The Annals of Statistics

Nonasymptotic bounds for autoregressive time series modeling

Alexander Goldenshluger and Assaf Zeevi

Full-text: Open access


The subject of this paper is autoregressive (AR) modeling of a stationary, Gaussian discrete time process, based on a finite sequence of observations. The process is assumed to admit an AR($\infty$) representation with exponentially decaying coefficients. We adopt the nonparametric minimax framework and study how well the process can be approximated by a finite­order AR model. A lower bound on the accuracy of AR approximations is derived, and a nonasymptotic upper bound on the accuracy of the regularized least squares estimator is established. It is shown that with a “proper” choice of the model order, this estimator is minimax optimal in order. These considerations lead also to a nonasymptotic upper bound on the mean squared error of the associated one­step predictor. A numerical study compares the common model selection procedures to the minimax optimal order choice.

Article information

Ann. Statist., Volume 29, Number 2 (2001), 417-444.

First available in Project Euclid: 24 December 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11]

Autoregressive approximation minimax risk rates of convergence


Goldenshluger, Alexander; Zeevi, Assaf. Nonasymptotic bounds for autoregressive time series modeling. Ann. Statist. 29 (2001), no. 2, 417--444. doi:10.1214/aos/1009210547.

Export citation


  • Akaike, H. (1970). Statistical predictor identification. Ann. Inst. Statist. Math. 21 243-247.
  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Control AC-19 716-723.
  • An, H.-Z., Chen, Z.-G. C. and Hannan, E. J. (1982). Autocorrelation, autoregression and autoregressive approximation. Ann. Statist. 10 926-936.
  • Berk, K. N. (1974). Consistent autoregressive spectral estimates. Ann. Statist. 2 489-502.
  • Bhansali, R. J. (1981). Effects of not knowing the order of an autoregressive process on the mean-squared error of prediction. J. Amer. Statist. Assoc. 76 588-597.
  • Bhansali, R. J. (1986). Asymptotically efficient selection of the order by the criterion autoregressive transfer function. Ann. Statist. 14 315-325.
  • Bosq, D. (1996). Nonparametric Statistics for Stochastic Processes. Springer, New York.
  • Bradley, R. C. (1986). Basic properties of strong mixing conditions. In Dependence in Probability and Statistics (E. Eberlein and M. Taqqu, eds.) 165-192. Birkh¨auser, Boston.
  • Brillinger, D. R. (1975). Times Series: Data Analysis and Theory. Holt, Rinehart and Winston, New York.
  • B ¨uhlmann, P. (1995). Moving average representations of autoregressive approximations. Stochastic Process. Appl. 60 331-342.
  • Doukhan, P. (1994). Mixing: Properties and Examples. Springer, New York.
  • Efromovich, S. (1998). Data-driven efficient estimation of the spectral density. J. Amer. Statist. Assoc. 92 762-769.
  • Goldenshluger, A. (1998). Nonparametric estimation of transfer functions: rates of convergence and adaptation. IEEE Trans. Inform. Theory 44 644-658.
  • Golubev, G. and Levit, B. (1996). Asymptotically efficient estimation for analytic distributions. Math. Methods Statist. 5 357-368.
  • Golubev, G. K., Levit, B. and Tsybakov A. (1996). Asymptotically efficient estimation of analytic functions in Gaussian noise. Bernoulli 2 167-181.
  • Gerencs´er, L. (1992). AR() estimation and non-parametric stochastic complexity. IEEE Trans. Inform Theory 38 1768-1778.
  • Grenander, U. and Szeg ¨o, G. (1984). Toeplitz Forms and Its Applications, 2nd ed. Chelsea, New York.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Hannan, E. J. and Kavalieris, L. (1986). Regression, autoregression models. J. Time Ser. Anal. 7 27-49.
  • Ibragimov, I. A. and Has'minskii, R. Z. (1981). Statistical Estimation. Springer, New York.
  • Ibragimov, I. A. and Rozanov, Y. A. (1978). Gaussian Random Processes. Springer, New York.
  • Korostelev, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Springer, New York.
  • Parzen, E. (1974). Some recent advances in time series modelling. IEEE Trans. Automat. Control AC-19 723-730.
  • Parzen, E. (1983). Autoregressive spectral estimation. In Time Series in the Frequency Domain 221-247. North-Holland, Amsterdam.
  • Rissanen, J. (1983). Universal coding, information, prediction, and estimation. IEEE Trans. Inform. Theory 30 629-636.
  • Rudin, W. (1974). Real and Complex Analysis, 2nd ed. McGraw-Hill, New York.
  • Saulis, L. and Statulevi cus, V. A. (1991). Limit Theorems for Large Deviations. Kluwer, Dordrecht.
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461-464.
  • Shibata, R. (1980). Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. Ann. Statist. 8 147-164.
  • Shibata, R. (1981). An optimal autoregressive spectral estimate. Ann. Statist. 9 300-306.