The Annals of Statistics

Estimation in nonlinear regression with Harris recurrent Markov chains

Degui Li, Dag Tjøstheim, and Jiti Gao

Full-text: Open access


In this paper, we study parametric nonlinear regression under the Harris recurrent Markov chain framework. We first consider the nonlinear least squares estimators of the parameters in the homoskedastic case, and establish asymptotic theory for the proposed estimators. Our results show that the convergence rates for the estimators rely not only on the properties of the nonlinear regression function, but also on the number of regenerations for the Harris recurrent Markov chain. Furthermore, we discuss the estimation of the parameter vector in a conditional volatility function, and apply our results to the nonlinear regression with $I(1)$ processes and derive an asymptotic distribution theory which is comparable to that obtained by Park and Phillips [Econometrica 69 (2001) 117–161]. Some numerical studies including simulation and empirical application are provided to examine the finite sample performance of the proposed approaches and results.

Article information

Ann. Statist., Volume 44, Number 5 (2016), 1957-1987.

Received: January 2014
Revised: August 2015
First available in Project Euclid: 12 September 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F12: Asymptotic properties of estimators
Secondary: 62M05: Markov processes: estimation

Asymptotic distribution asymptotically homogeneous function $\beta$-null recurrent Markov chain Harris recurrence integrable function least squares estimation nonlinear regression


Li, Degui; Tjøstheim, Dag; Gao, Jiti. Estimation in nonlinear regression with Harris recurrent Markov chains. Ann. Statist. 44 (2016), no. 5, 1957--1987. doi:10.1214/15-AOS1379.

Export citation


  • Bandi, F. and Phillips, P. C. B. (2009). Nonstationary continuous-time processes. In Handbook of Financial Econometrics (Y. Aït-Sahalia and L. P. Hansen, eds.) 1 139–201.
  • Bec, F., Rahbek, A. and Shephard, N. (2008). The ACR model: A multivariate dynamic mixture autoregression. Oxf. Bull. Econ. Stat. 70 583–618.
  • Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • Chan, N. and Wang, Q. (2012). Nonlinear cointegrating regressions with nonstationary time series. Working paper, School of Mathematics, Univ. Sydney.
  • Chen, X. (1999). How often does a Harris recurrent Markov chain recur? Ann. Probab. 27 1324–1346.
  • Chen, X. (2000). On the limit laws of the second order for additive functionals of Harris recurrent Markov chains. Probab. Theory Related Fields 116 89–123.
  • Chen, L.-H., Cheng, M.-Y. and Peng, L. (2009). Conditional variance estimation in heteroscedastic regression models. J. Statist. Plann. Inference 139 236–245.
  • Chen, J., Gao, J. and Li, D. (2012). Estimation in semi-parametric regression with non-stationary regressors. Bernoulli 18 678–702.
  • Gao, J. (2007). Nonlinear Time Series: Semiparametric and Nonparametric Methods. Monographs on Statistics and Applied Probability 108. Chapman & Hall/CRC, Boca Raton, FL.
  • Gao, J., Tjøstheim, D. and Yin, J. (2013). Estimation in threshold autoregressive models with a stationary and a unit root regime. J. Econometrics 172 1–13.
  • Gao, J., Kanaya, S., Li, D. and Tjøstheim, D. (2015). Uniform consistency for nonparametric estimators in null recurrent time series. Econometric Theory 31 911–952.
  • Han, H. and Kristensen, D. (2014). Asymptotic theory for the QMLE in GARCH-X models with stationary and nonstationary covariates. J. Bus. Econom. Statist. 32 416–429.
  • Han, H. and Park, J. Y. (2012). ARCH/GARCH with persistent covariate: Asymptotic theory of MLE. J. Econometrics 167 95–112.
  • Höpfner, R. and Löcherbach, E. (2003). Limit theorems for null recurrent Markov processes. Mem. Amer. Math. Soc. 161 vi+92.
  • Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Ann. Math. Statist. 40 633–643.
  • Kallianpur, G. and Robbins, H. (1954). The sequence of sums of independent random variables. Duke Math. J. 21 285–307.
  • Karlsen, H. A., Myklebust, T. and Tjøstheim, D. (2007). Nonparametric estimation in a nonlinear cointegration type model. Ann. Statist. 35 252–299.
  • Karlsen, H. A., Myklebust, T. and Tjøstheim, D. (2010). Nonparametric regression estimation in a null recurrent time series. J. Statist. Plann. Inference 140 3619–3626.
  • Karlsen, H. A. and Tjøstheim, D. (2001). Nonparametric estimation in null recurrent time series. Ann. Statist. 29 372–416.
  • Kasahara, Y. (1984). Limit theorems for Lévy processes and Poisson point processes and their applications to Brownian excursions. J. Math. Kyoto Univ. 24 521–538.
  • Kristensen, D. and Rahbek, A. (2013). Testing and inference in nonlinear cointegrating vector error correction models. Econometric Theory 29 1238–1288.
  • Lai, T. L. (1994). Asymptotic properties of nonlinear least squares estimates in stochastic regression models. Ann. Statist. 22 1917–1930.
  • Li, R. and Nie, L. (2008). Efficient statistical inference procedures for partially nonlinear models and their applications. Biometrics 64 904–911.
  • Li, D., Tjøstheim, D. and Gao, J. (2015). Supplement to “Estimation in nonlinear regression with Harris recurrent Markov chains.” DOI:10.1214/15-AOS1379SUPP.
  • Lin, Z., Li, D. and Chen, J. (2009). Local linear $M$-estimators in null recurrent time series. Statist. Sinica 19 1683–1703.
  • Ling, S. (2007). Self-weighted and local quasi-maximum likelihood estimators for ARMA-GARCH/IGARCH models. J. Econometrics 140 849–873.
  • Lu, Z. (1998). On the geometric ergodicity of a non-linear autoregressive model with an autoregressive conditional heteroscedastic term. Statist. Sinica 8 1205–1217.
  • Malinvaud, E. (1970). The consistency of nonlinear regressions. Ann. Math. Statist. 41 956–969.
  • Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd ed. Cambridge Univ. Press, Cambridge.
  • Myklebust, T., Karlsen, H. A. and Tjøstheim, D. (2012). Null recurrent unit root processes. Econometric Theory 28 1–41.
  • Nummelin, E. (1984). General Irreducible Markov Chains and Nonnegative Operators. Cambridge Tracts in Mathematics 83. Cambridge Univ. Press, Cambridge.
  • Park, J. Y. and Phillips, P. C. B. (1999). Asymptotics for nonlinear transformations of integrated time series. Econometric Theory 15 269–298.
  • Park, J. Y. and Phillips, P. C. B. (2001). Nonlinear regressions with integrated time series. Econometrica 69 117–161.
  • Peng, L. and Yao, Q. (2003). Least absolute deviations estimation for ARCH and GARCH models. Biometrika 90 967–975.
  • Schienle, M. (2011). Nonparametric nonstationary regression with many covariates. Working paper, Humboldt–Univ. Berlin.
  • Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
  • Skouras, K. (2000). Strong consistency in nonlinear stochastic regression models. Ann. Statist. 28 871–879.
  • Teräsvirta, T., Tjøstheim, D. and Granger, C. W. J. (2010). Modelling Nonlinear Economic Time Series. Oxford Univ. Press, Oxford.
  • Wu, C.-F. (1981). Asymptotic theory of nonlinear least squares estimation. Ann. Statist. 9 501–513.

Supplemental materials

  • Supplement to “Estimation in nonlinear regression with Harris recurrent Markov chains”. We provide some additional simulation studies, the detailed proofs of the main results in Section 3, the proofs of Lemmas A.1 and A.2 and Theorem 4.1.