Electronic Journal of Statistics

Assessing the multivariate normal approximation of the maximum likelihood estimator from high-dimensional, heterogeneous data

Andreas Anastasiou

Full-text: Open access

Abstract

The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain upper bounds in cases where the MLE cannot be expressed analytically.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 3794-3828.

Dates
Received: August 2017
First available in Project Euclid: 30 November 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1543568429

Digital Object Identifier
doi:10.1214/18-EJS1492

Subjects
Primary: 62F12: Asymptotic properties of estimators
Secondary: 62E17: Approximations to distributions (nonasymptotic)

Keywords
Multi-parameter maximum likelihood estimation multivariate normal approximation Stein’s method

Rights
Creative Commons Attribution 4.0 International License.

Citation

Anastasiou, Andreas. Assessing the multivariate normal approximation of the maximum likelihood estimator from high-dimensional, heterogeneous data. Electron. J. Statist. 12 (2018), no. 2, 3794--3828. doi:10.1214/18-EJS1492. https://projecteuclid.org/euclid.ejs/1543568429


Export citation

References

  • Anastasiou, A. (2017). Bounds for the normal approximation of the maximum likelihood estimator from $m-$dependent random variables., Statistics & Probability Letters, 129, 171–181.
  • Anastasiou, A. & Ley, C. (2017). Bounds for the asymptotic normality of the maximum likelihood estimator using the Delta method., ALEA, Lat. Am. J. Probab. Math. Stat., 14, 153–171.
  • Anastasiou, A. & Reinert, G. (2017). Bounds for the normal approximation of the maximum likelihood estimator., Bernoulli, 23, 191–218.
  • Berk, R. H. (1972). Consistency and asymptotic normality of MLE’s for exponential models., The Annals of Mathematical Statistics, 43, 193–204.
  • Billingsley, P. (1961). Statistical Methods in Markov Chains., The Annals of Mathematical Statistics, 32, No.1, 12–40.
  • Casella, G. & Berger, R. L. (2002)., Statistical Inference. Brooks/Cole, Cengage Learning, Duxbury, Pacific Grove, second edition.
  • Davison, A. C. (2008)., Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, first edition.
  • Fahrmeir, L. & Kaufmann, H. (1985). Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models., The Annals of Statistics, 13, No.1, 342–368.
  • Gorham, J., Duncan, A. B., Vollmer, S. J., & Mackey, L. (2016). Measuring sample quality with diffusions., https://arxiv.org/pdf/1611.06972.pdf.
  • Hoadley, B. (1971). Asymptotic Properties of Maximum Likelihood Estimators for the Independent Not Identically Distributed Case., The Annals of Mathematical Statistics, 42, No.6, 1977–1991.
  • Kiefer, J. C. (1968). Statistical inference. In, The future of statistics. Proceedings of a Conference on the Future of Statistics held at the University of Wisconsin, Madison, Wisconsin, June 1967, pages 139–142. Academic Press, New York-London.
  • Koroljuk, V. S. & Borovskich, Y. V. (1994)., Theory of U-statistics. Mathematics and its Applications 273. Kluwer Academic Publishers Group, Dordrecht. Translated from the 1989 Russian original by P.V. Malyshev and D.V. Malyshev and revised by the authors.
  • Lauritzen, S. (1988)., Extremal Families and Systems of Sufficient Statistics. Lecture Notes in Statistics, No.49. Springer-Verlag, Berlin-Heidelberg-New York.
  • Lauritzen, S. (1996)., Graphical Models. Oxford: Clarendon Press.
  • Mäkeläinen, T., Schmidt, K., & Styan, G. P. H. (1981). On the existence and uniqueness of the maximum likelihood estimate of a vector-valued parameter in fixed-size samples., The Annals of Statistics, 9, No.4, 758–767.
  • Massam, H., Li, Q., & Gao, X. (2018). Bayesian precision and covariance matrix estimation for graphical Gaussian models with edge and vertex symmetries., Biometrika, 105, 371–388.
  • Pinelis, I. (2017). Optimal-order uniform and nonuniform bounds on the rate of convergence to normality for maximum likelihood estimators., Electronic Journal of Statistics, 11, 1160–1179.
  • Pinelis, I. & Molzon, R. (2016). Optimal-order bounds on the rate of convergence to normality in the multivariate delta method., Electronic Journal of Statistics, 10, 1001–1063.
  • Reinert, G. & Röllin, A. (2009). Multivariate normal approximation with Stein’s method of exchangeable pairs under a general linearity condition., The Annals of Probability, 37, No.6, 2150–2173.
  • Stein, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In, Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, volume 2, pages 586–602. Berkeley: University of California Press.