Translator Disclaimer
August 2022 The asymptotic distribution of the MLE in high-dimensional logistic models: Arbitrary covariance
Qian Zhao, Pragya Sur, Emmanuel J. Candès
Author Affiliations +
Bernoulli 28(3): 1835-1861 (August 2022). DOI: 10.3150/21-BEJ1401

Abstract

We study the distribution of the maximum likelihood estimate (MLE) in high-dimensional logistic models, where covariates are Gaussian with an arbitrary covariance structure. We prove that in the limit of large problems holding the ratio between the number p of covariates and the sample size n constant, every finite list of MLE coordinates follows a multivariate normal distribution. Concretely, the jth coordinate βˆj of the MLE is asymptotically normally distributed with mean αβj and standard deviation στj; here, βj is the value of the true regression coefficient, and τj the standard deviation of the jth predictor conditional on all the others. The numerical parameters α>1 and σ only depend upon the problem dimensionality pn and the overall signal strength, and can be accurately estimated. Our results imply that the MLE’s magnitude is biased upwards and that the MLE’s standard deviation is greater than that predicted by classical theory. We present a series of experiments on simulated and real data showing excellent agreement with the theory.

Acknowledgements

E. C. was supported by the National Science Foundation via DMS 1712800 and via the Stanford Data Science Collaboratory OAC 1934578, and by a generous gift from TwoSigma. P.S. was supported by the Center for Research on Computation and Society, Harvard John A. Paulson School of Engineering and Applied Sciences. Q. Z. would like to thank Stephen Bates for helpful comments about an early version of this paper.

Citation

Download Citation

Qian Zhao. Pragya Sur. Emmanuel J. Candès. "The asymptotic distribution of the MLE in high-dimensional logistic models: Arbitrary covariance." Bernoulli 28 (3) 1835 - 1861, August 2022. https://doi.org/10.3150/21-BEJ1401

Information

Received: 1 October 2020; Published: August 2022
First available in Project Euclid: 25 April 2022

Digital Object Identifier: 10.3150/21-BEJ1401

Keywords: Gaussian covariates , high-dimensional inference , logistic regression , maximum likelihood estimation

JOURNAL ARTICLE
27 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.28 • No. 3 • August 2022
Back to Top