Electronic Journal of Statistics

Optimal-order uniform and nonuniform bounds on the rate of convergence to normality for maximum likelihood estimators

Iosif Pinelis

Full-text: Open access

Abstract

It is well known that, under general regularity conditions, the distribution of the maximum likelihood estimator (MLE) is asymptotically normal. Very recently, bounds of the optimal order $O(1/\sqrt{n})$ on the closeness of the distribution of the MLE to normality in the so-called bounded Wasserstein distance were obtained [2, 1], where $n$ is the sample size. However, the corresponding bounds on the Kolmogorov distance were only of the order $O(1/n^{1/4})$. In this paper, bounds of the optimal order $O(1/\sqrt{n})$ on the closeness of the distribution of the MLE to normality in the Kolmogorov distance are given, as well as their nonuniform counterparts, which work better in tail zones of the distribution of the MLE. These results are based in part on previously obtained general optimal-order bounds on the rate of convergence to normality in the multivariate delta method. The crucial observation is that, under natural conditions, the MLE can be tightly enough bracketed between two smooth enough functions of the sum of independent random vectors, which makes the delta method applicable. It appears that the nonuniform bounds for MLEs in general have no precedents in the existing literature; a special case was recently treated by Pinelis and Molzon [20]. The results can be extended to $M$-estimators.

Article information

Source
Electron. J. Statist., Volume 11, Number 1 (2017), 1160-1179.

Dates
Received: July 2016
First available in Project Euclid: 11 April 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1491897618

Digital Object Identifier
doi:10.1214/17-EJS1264

Mathematical Reviews number (MathSciNet)
MR3634332

Zentralblatt MATH identifier
1361.62013

Subjects
Primary: 62F10: Point estimation 62F12: Asymptotic properties of estimators 60F05: Central limit and other weak theorems 60E15: Inequalities; stochastic orderings

Keywords
Maximum likelihood estimators Berry–Esseen bounds delta method rates of convergence

Rights
Creative Commons Attribution 4.0 International License.

Citation

Pinelis, Iosif. Optimal-order uniform and nonuniform bounds on the rate of convergence to normality for maximum likelihood estimators. Electron. J. Statist. 11 (2017), no. 1, 1160--1179. doi:10.1214/17-EJS1264. https://projecteuclid.org/euclid.ejs/1491897618


Export citation

References

  • [1] Anastasiou, A. and Ley, C. (2015). New simpler bounds to assess the asymptotic normality of the maximum likelihood estimator., http://arxiv.org/abs/1508.04948.
  • [2] Anastasiou, A. and Reinert, G. (2017). Bounds for the normal approximation of the maximum likelihood estimator., Bernoulli 23 191–218.
  • [3] Bentkus, V., Bloznelis, M. and Götze, F. (1996). A Berry-Esséen bound for Student’s statistic in the non-i.i.d. case., J. Theoret. Probab. 9 765–796.
  • [4] Bentkus, V. and Götze, F. (1996). The Berry-Esseen bound for Student’s statistic., Ann. Probab. 24 491–503.
  • [5] Borovkov, A. A. (1998)., Mathematical statistics. Gordon and Breach Science Publishers, Amsterdam Translated from the Russian by A. Moullagaliev and revised by the author.
  • [6] Chen, L. H. Y. and Shao, Q.-M. (2007). Normal approximation for nonlinear statistics using a concentration inequality approach., Bernoulli 13 581–599.
  • [7] Ibragimov, I. A. and Radavichyus, M. È. (1981). On large deviation probabilities for maximum likelihood estimators., Dokl. Akad. Nauk SSSR 257 1048–1052.
  • [8] Keilson, J. (1979)., Markov chain models—rarity and exponentiality. Applied Mathematical Sciences 28. Springer-Verlag, New York-Berlin.
  • [9] Kiefer, J. C. (1968). Statistical inference. In, The future of statistics. Proceedings of a Conference on the Future of Statistics held at the University of Wisconsin, Madison, Wisconsin, June 1967, 139–142. Academic Press, New York-London.
  • [10] Koroljuk, V. S. and Borovskich, Y. V. (1994)., Theory of $U$-statistics. Mathematics and its Applications 273. Kluwer Academic Publishers Group, Dordrecht. Translated from the 1989 Russian original by P. V. Malyshev and D. V. Malyshev and revised by the authors.
  • [11] Kourouklis, S. (1984). A large deviation result for the likelihood ratio statistic in exponential families., Ann. Statist. 12 1510–1521.
  • [12] Lehmann, E. L. and Casella, G. (1998)., Theory of point estimation, second ed. Springer Texts in Statistics. Springer-Verlag, New York.
  • [13] Miao, Y. (2010). Concentration inequality of maximum likelihood estimator., Appl. Math. Lett. 23 1305–1309.
  • [14] Michel, R. and Pfanzagl, J. (1971). The accuracy of the normal approximation for minimum contrast estimates., Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 18 73–84.
  • [15] Mogul’skiĭ, A. A. (1988). Large deviations for the maximum likelihood estimators. In, Probability theory and mathematical statistics (Kyoto, 1986). Lecture Notes in Math. 1299 326–331. Springer, Berlin.
  • [16] Pfanzagl, J. (1971). The Berry-Esseen bound for minimum contrast estimates., Metrika 17 82–91.
  • [17] Pfanzagl, J. (1972/73). The accuracy of the normal approximation for estimates of vector parameters., Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 25 171–198.
  • [18] Pinelis, I. (2015). Exact Rosenthal-type bounds., Ann. Probab. 43 2511–2544.
  • [19] Pinelis, I. (2016). Optimal-order bounds on the rate of convergence to normality for maximum likelihood estimators., http://arxiv.org/abs/1601.02177.
  • [20] Pinelis, I. and Molzon, R. (2016). Optimal-order bounds on the rate of convergence to normality in the multivariate delta method., Electron. J. Stat. 10 1001–1063.
  • [21] Radavichyus, M. È. (1983). Probabilities of large deviations for maximum likelihood estimators., Dokl. Akad. Nauk SSSR 268 551–556.
  • [22] Radavičjus, M. È. (1981). Probabilities of large and moderate deviations for maximum likelihood estimates., Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 108 154–169, 196, 199. Studies in mathematical statistics, V.