The Annals of Statistics

Conditional least squares estimation in nonstationary nonlinear stochastic regression models

Christine Jacob

Full-text: Open access


Let {Zn} be a real nonstationary stochastic process such that $E(Z_{n}|{\mathcal{F}}_{n-1})\stackrel{\mathrm{a.s.}}<\infty$ and $E(Z_{n}^{2}|{\mathcal{F}}_{n-1})\stackrel{\mathrm{a.s.}}<\infty$, where $\{{\mathcal{F}}_{n}\}$ is an increasing sequence of σ-algebras. Assuming that $E(Z_{n}|{\mathcal{F}}_{n-1})=g_{n}(\theta_{0},\nu_{0})=g_{n}^{(1)}(\theta_{0})+g_{n}^{(2)}(\theta _{0},\nu_{0})$, θ0∈ℝp, p<∞, ν0∈ℝq and q≤∞, we study the asymptotic properties of $\widehat{\theta}_{n}:=\arg\min_{\theta}\sum_{k=1}^{n}(Z_{k}-g_{k}({\theta,\widehat{\nu}}))^{2}\lambda _{k}^{-1}$, where λk is ${\mathcal{F}}_{k-1}$-measurable, ̂ν={̂νk} is a sequence of estimations of ν0, gn(θ, ̂ν) is Lipschitz in θ and gn(2)(θ0, ̂ν)−gn(2)(θ, ̂ν) is asymptotically negligible relative to gn(1)(θ0)−gn(1)(θ). We first generalize to this nonlinear stochastic model the necessary and sufficient condition obtained for the strong consistency of {̂θn} in the linear model. For that, we prove a strong law of large numbers for a class of submartingales. Again using this strong law, we derive the general conditions leading to the asymptotic distribution of ̂θn. We illustrate the theoretical results with examples of branching processes, and extension to quasi-likelihood estimators is also considered.

Article information

Ann. Statist., Volume 38, Number 1 (2010), 566-597.

First available in Project Euclid: 31 December 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 62J02: General nonlinear regression 62F12: Asymptotic properties of estimators 62M05: Markov processes: estimation 62M09: Non-Markovian processes: estimation 62P05: Applications to actuarial sciences and financial mathematics 62P10: Applications to biology and medical sciences
Secondary: 60G46: Martingales and classical analysis 60F15: Strong theorems

Stochastic nonlinear regression heteroscedasticity nonstationary process time series branching process conditional least squares estimator quasi-likelihood estimator consistency asymptotic distribution martingale difference submartingale polymerase chain reaction


Jacob, Christine. Conditional least squares estimation in nonstationary nonlinear stochastic regression models. Ann. Statist. 38 (2010), no. 1, 566--597. doi:10.1214/09-AOS733.

Export citation


  • [1] Anderson, T. W. and Taylor, J. B. (1979). Strong consistency of least squares estimates in dynamic models. Ann. Statist. 7 484–489.
  • [2] Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • [3] Dion, J.-P. (1974). Estimation of the mean and the initial probabilities of a branching process. J. Appl. Probab. 11 687–694.
  • [4] Dacunha-Castelle, D. and Duflo, M. (1993). Probabilités et Statistiques. 2. Problèmes à Temps Mobile. Masson, Paris.
  • [5] Engle, R. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica 50 987–1007.
  • [6] Feigin, P. D. (1977). A note on maximum likelihood estimation for simple branching processes. Austral. J. Statist. 19 152–154.
  • [7] Godambe, V. P. and Heyde, C. C. (1987). Quasi-likelihood and optimal estimation. Internat. Statist. Rev. 55 231–244.
  • [8] Guttorp, P. (1991). Statistical Inference for Branching Processes. Wiley, New York.
  • [9] Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Wiley, New York.
  • [10] Hall, P. and Yao, Q. (2003). Inference in ARCH and GARCH models with heavy-tailed errors. Econometrica 71 285–317.
  • [11] Harris, T. E. (1948). Branching processes. Ann. Math. Statist. 19 474–494.
  • [12] Hutton, J. E. and Nelson, P. I. (1986). Quasilikelihood estimation for semimartingales. Stochastic Process. Appl. 22 245–257.
  • [13] Jacob, C. and Peccoud, J. (1998). Estimation of the parameters of a branching process from migrating binomial observations. Adv. in Appl. Probab. 30 948–967.
  • [14] Jacob, C. (2008). Conditional least squares estimation in nonlinear and nonstationary stochastic regression models: Asymptotic properties and examples. Technical Report UR341, INRA, Jouy-en-Josas, France.
  • [15] Lalam, N. and Jacob, C. (2004). Estimation of the offspring mean in a supercritical or near-critical size-dependent branching process. Adv. in Appl. Probab. 36 582–601.
  • [16] Jacob, C., Lalam, N. and Yanev, N. (2005). Statistical inference for processes depending on exogenous inputs and application in regenerative processes. Pliska Stud. Math. Bulgar. 17 109–136.
  • [17] Jagers, P. and Klebaner, F. C. (2003). Random variation and concentration effects in PCR. J. Theoret. Biol. 224 299–304.
  • [18] Jennrich, R. I. (1969). Asymptotic properties of nonlinear least squares estimators. Ann. Math. Statist. 40 633–643.
  • [19] Klebaner, F. C. (1984). On population-size dependent branching processes. Adv. in Appl. Probab. 16 30–55.
  • [20] Lai, T. L., Robbins, H. and Wei, C. Z. (1979). Strong consistency of estimates in multiple regression. II. J. Multivariate Anal. 9 343–361.
  • [21] Lai, T. L. and Wei, C. Z. (1982). Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. Ann. Statist. 10 154–166.
  • [22] Lai, T. L. and Wei, C. Z. (1983). Asymptotic properties of general autoregressive models and strong consistency of least-squares estimates of their parameters, J. Multivariate Anal. 13 1–23.
  • [23] Lai, T. L. (1994). Asymptotic properties of nonlinear least squares estimates in stochastic regression models. Ann. Statist. 22 1917–1930.
  • [24] Lalam, N., Jacob, C. and Jagers, P. (2004). Modelling of the PCR amplification process by a size-dependent branching process and estimation of the efficiency. Adv. in Appl. Probab. 36 602–615.
  • [25] Maaouia, F. and Touati, A. (2005). Identification of multitype branching processes. Ann. Statist. 33 2655–2694.
  • [26] Ngatchou-Wandji, J. (2008). Estimation in a class of nonlinear heteroscedastic time-series models. Electron. J. Stat. 2 40–62.
  • [27] Peccoud, J. and Jacob, C. (1996). Theoretical uncertainty of measurements using quantitative polymerase chain reaction. Biophysical J. 71 101–108.
  • [28] Rahimov, I. (1995). Random Sums and Branching Stochastic Processes. Lecture Notes in Statistics 96 195. Springer, New York.
  • [29] Rebolledo, R. (1980). Central limit theorems for local martingales. Z. Wahrsch. Verw. Gebiete 51 269–286.
  • [30] Schnell, S. and Mendoza, C. (1997). Enzymological considerations for a theoretical description of the quantitative competitive polymerase chain reaction. J. Theoret. Biol. 184 433–440.
  • [31] Skouras, K. (2000). Strong consistency in nonlinear stochastic regression models. Ann. Statist. 28 871–879.
  • [32] Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models, and the Gauss–Newton method. Biometrika 61 439–447.
  • [33] Wei, C. Z. (1985). Asymptotic properties of least-squares estimates in stochastic regression models. Ann. Statist. 13 1498–1508.
  • [34] Wu, C. F. (1981). Asymptotic theory of nonlinear least squares estimation. Ann. Statist. 9 501–513.
  • [35] Yanev, N. (2009). Statistical inference for branching processes. In Records and Branching Processes (M. Ahsanullah and G. Yanev, eds.). Nova Science Publishers, New York.
  • [36] Yao, J. F. (2000). On least squares estimation for stable nonlinear AR processes. Ann. Inst. Statist. Math. 52 316–331.