The class of nonlinear stochastic regression models includes most of the linear and nonlinear models used in time series, stochastic control and stochastic approximation schemes.The consistency of least squares estimators was established first by Lai.We present another set of sufficient conditions for consistency, which avoid the use of partial derivatives and are closer in spirit to the conditions presented by Wu for non-stochastic regression models with independent errors.
"Strong consistency in nonlinear stochastic regression models." Ann. Statist. 28 (3) 871 - 879, June 2000. https://doi.org/10.1214/aos/1015952002