Electronic Journal of Statistics

Recursive construction of confidence regions

Abstract

Assuming that one-step transition kernel of a discrete time, time-homogenous Markov chain model is parameterized by a parameter $\theta\in\boldsymbol{\Theta}$, we derive a recursive (in time) construction of confidence regions for the unknown parameter of interest, say $\theta^{*}\in\boldsymbol{\Theta}$. It is supposed that the observed data used in the construction of the confidence regions is generated by a Markov chain whose transition kernel corresponds to $\theta^{*}$. The key step in our construction is the derivation of a recursive scheme for an appropriate point estimator of $\theta^{*}$. To achieve this, we start by what we call the base recursive point estimator, using which we design a quasi-asymptotically linear recursive point estimator (a concept introduced in this paper). For the latter estimator we prove its weak consistency and asymptotic normality. The recursive construction of confidence regions is needed not only for the purpose of speeding up the computation of the successive confidence regions, but, primarily, for the ability to apply the dynamic programming principle in the context of robust adaptive stochastic control methodology.

Article information

Source
Electron. J. Statist., Volume 11, Number 2 (2017), 4674-4700.

Dates
First available in Project Euclid: 18 November 2017

https://projecteuclid.org/euclid.ejs/1510974130

Digital Object Identifier
doi:10.1214/17-EJS1362

Mathematical Reviews number (MathSciNet)
MR3724972

Zentralblatt MATH identifier
06816629

Citation

Bielecki, Tomasz; Chen, Tao; Cialenco, Igor. Recursive construction of confidence regions. Electron. J. Statist. 11 (2017), no. 2, 4674--4700. doi:10.1214/17-EJS1362. https://projecteuclid.org/euclid.ejs/1510974130

References

• [1] D. J. Aldous and G. K. Eagleson. On Mixing and Stability of Limit Theorems., The Annals of Probability, 6(2):325–331, April 1978.
• [2] T. R. Bielecki, T. Chen, I. Cialenco, A. Cousin, and M. Jeanblanc. Adaptive Robust Control Under Model Uncertainty. Preprint, 2017.
• [3] R. Buche and H. J. Kushner. Rate of convergence for constrained stochastic approximation algorithms., SIAM J. Control Optim., 40(4) :1011–1041, 2001/02.
• [4] I. Crimaldi and L. Pratelli. Convergence Result for Multivariate Martingales., Stoch. Proc. and their Applic., 115:571–577, 2005.
• [5] V. Fabian. On Asymptotic Normality in stochastic approximation., Ann. Math. Statist., 39(4) :1327–1332, 1968.
• [6] R. A. Fisher. Theory of statistical estimation., Proc. Cambridge Philos. Soc., 22:700–725, 1925.
• [7] E. Häusler and H. Luschgy., Stable Convergence and Stable Limit Theorems. Springer, 2015.
• [8] O. Hernández-Lerma and J. Lasserre. Further criteria for positive Harris recurrence of Markov chains., Proceedings of the American Mathematical Society, 129(5) :1521–1524, 2000.
• [9] H. Kushner and D. Clark., Stochastic Approximation Methods for Constrained and Unconstrained Systems, volume 26 of Applied Mathematical Sciences. Springer-Verlag New York, 1978.
• [10] S. Kullback and R. A. Leibler. On information and sufficiency., Annals of Mathematical Statistics, 22(1):79–86, 1951.
• [11] J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function., The Annals of Mathematical Statistics, 23(3):462–466, 1952.
• [12] H. Kushner and G. Yin., Stochastic Approximation and Recursive Algorithms and Applications, volume 35 of Stochastic Modelling and Applied Probability. Springer-Verlag New York, 2003.
• [13] L. LeCam. On the asymptotic theory of estimation and testing hypothesis., Proccedings in the Third Berkely Symposium on Mathematical Statist. and Probab., 1:129–156, 1956.
• [14] L. LeCam. Locally asymptotic normal families of distributions., Univ. California Publ. in Statist., 3(2):37–98, 1960.
• [15] T. L. Lai and H. Robbins. Adaptive design and stochastic approximation., Ann. Statist., 7(6) :1196–1221, 1979.
• [16] L. Ljung and Söderström., Theory and Practice of Recursive Identification, volume 4 of The MIT Press Series in Signal Processing, Optimization, and Control. MIT Press, 1987.
• [17] S. Meyn and R. Tweedie., Markov Chains and Stochastic Stability. Cambridge Mathematical Library. Cambridge University Press, 1993.
• [18] D. Revuz., Markov Chains, volume 11 of North-Holland Mathematical Library. Elsevier Science Publishers B.V., revised edition edition, 1984.
• [19] H. Robbins and S. Monro. A stochastic approximation method., The Annals of Mathematical Statistics, 22(3):400–407, 1951.
• [20] J. Sacks. Asymptotic distribution of stochastic approximation procedures., Ann. Math. Statist., 29(2):373–405, 1958.
• [21] T. Sharia. Recursive parameter estimation: asymptotic expansion., Annals of the Institute of Statistical Mathematics, 62(2):343–362, 2010.
• [22] A. Shiryayev., Probability, volume 95 of Graduate Texts in Mathematics. Springer New York, 1984.
• [23] T. Sharia and L. Zhong. Rate of convergence of truncated stochastic approximation procedures with moving bounds., Mathematical Methods of Statistics, 25(4):262–280, 2016.
• [24] G. Yin. A stopping rule for least-squares identification., IEEE Trans. Automat. Control, 34(6), 1989.
• [25] G. Yin. A stopping rule for the robbins-monro method., Journal of Optimization Theory and Applications, 67:151–173, 1990.