Electronic Journal of Statistics

Critical dimension in profile semiparametric estimation

Andreas Andresen and Vladimir Spokoiny

Full-text: Open access

Abstract

This paper revisits the classical inference results for profile quasi maximum likelihood estimators (profile MLE) in semiparametric models. We mainly focus on two prominent theorems: the Wilks phenomenon and Fisher expansion for the profile MLE are stated in a new fashion allowing finite samples and model misspecification. The method of study is also essentially different from the usual analysis of the semiparametric problem based on the notion of the hardest parametric submodel. Instead we derive finite sample deviation bounds for the linear approximation error for the gradient of the loglikelihood. This novel approach particularly allows to address the impact of the effective target and nuisance dimension on the accuracy of the results. The obtained nonasymptotic results are surprisingly sharp and yield the classical asymptotic statements including the asymptotic normality and efficiency of the profile MLE. The general results are specified for the important special case of an i.i.d. sample and the analysis is exemplified with a single index model.

Article information

Source
Electron. J. Statist., Volume 8, Number 2 (2014), 3077-3125.

Dates
First available in Project Euclid: 15 January 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1421330631

Digital Object Identifier
doi:10.1214/14-EJS982

Mathematical Reviews number (MathSciNet)
MR3301302

Zentralblatt MATH identifier
1308.62031

Subjects
Primary: 62F10: Point estimation
Secondary: 62J12: Generalized linear models 62F25: Tolerance and confidence regions 62H12: Estimation

Keywords
Profile maximum likelihood local linear approximation spread local concentration

Citation

Andresen, Andreas; Spokoiny, Vladimir. Critical dimension in profile semiparametric estimation. Electron. J. Statist. 8 (2014), no. 2, 3077--3125. doi:10.1214/14-EJS982. https://projecteuclid.org/euclid.ejs/1421330631


Export citation

References

  • [1] Andresen, A., Finite sample analysis of profile m-estimation in the single index model. ArXiv :1406.4052, 2014.
  • [2] Andresen, A., A note on critical dimensions in profile semiparametric estimation. ArXiv :1410.4709, 2014.
  • [3] Andresen, A., A note on the bias of sieve profile estimation. ArXiv: 1406.4045, 2014.
  • [4] Berry, A., The accuracy of the gaussian approximation to the sum of independent variates., Transactions of the American Mathematical Society, 49(1):122–136, 1941.
  • [5] Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A., Efficient and Adaptive Estimation for Semiparametric Models. Springer, 1998.
  • [6] Boucheron, S. and Massart, P., A high-dimensional Wilks phenomenon., Probability Theory and Related Fields, 150:405–433, 2011. 10.1007/s00440-010-0278-7.
  • [7] Delecroix, M., Haerdle, W., and Hristache, M., Efficient estimation in single-index regression. Technical report, SFB 373, Humboldt Univ. Berlin, 1997.
  • [8] Fan, J. and Huang, T., Profile likelihood inferences on semiparametric varying-coefficient partially linear models., Bernoulli, 11(6) :1031–1057, 2005.
  • [9] Fan, J., Zhang, C., and Zhang, J., Generalized likelihood ratio statistics and Wilks phenomenon., Ann. Stat., 29(1):153–193, 2001.
  • [10] Ghosal, S., Asymptotic normality of posterior distributions in high-dimensional linear models., Bernoulli, 5(2):315–331, 1999.
  • [11] Ghosal, S., Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity., J. Multivariate Anal., 74(1):49–68, 2000.
  • [12] Haerdle, W., Hall, P., and Ichimura, H., Optimal smoothing in single-index models., Ann. Statist., 21:157–178, 1993.
  • [13] Hall, P., The Bootstrap and Edgeworth Expansion. Springer, 1992.
  • [14] Huber, P. J., The behavior of maximum likelihood estimates under nonstandard conditions. Proc. 5th Berkeley Symp. Math. Stat. Probab., Univ. Calif. 1965/1966, 1, 221–233, (1967).
  • [15] Ibragimov, I. A. and Khas’minskij, R. Z., Statistical Estimation. Asymptotic Theory. Transl. from the Russian by Samuel Kotz. Springer-Verlag, New York–Heidelberg–Berlin, 1981.
  • [16] Ichimura, H., Semiparametric least squares (sls) and weighted sls estimation of single-index models., J. Econometrics, 58:71–120, 1993.
  • [17] Kim, Y., The Bernstein-von Mises theorem for the proportional hazard model., Ann. Statist., 34(4) :1678–1700, 2006.
  • [18] Kosorok, M. R., Introduction to Empirical Processes and Semiparametric Inference. Springer in Statistics, 2005.
  • [19] Mammen, E., Asymptotics with increasing dimension for robust regression with applications to the bootstrap., Ann. Stat., 17(1):382–400, 1989.
  • [20] Mammen, E., Bootstrap and wild bootstrap for high dimensional linear models., Ann. Stat., 21(1):255–285, 1993.
  • [21] Mammen, E., Empirical process of residuals for high-dimensional linear models., Ann. Stat., 24(1):307–335, 1996.
  • [22] Murphy, S. A. and van der Vaart, A. W., On profile likelihood., Journal of the American Statistical Association, 95(450):449–465, 2000.
  • [23] Murphy, S. A. and van der Vaart, A. W., Observed information in semi-parametric models., Bernoulli, 5(3):381–412, 1999.
  • [24] Newey, W. K., Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79(1):147–168, 1997.
  • [25] Portnoy, S., Asymptotic behavior of m estimators of p regression parameters when $p^2/n$ is large: Ii normal approximation., The Annals of Statistics, 13(4) :1403–1417, 1985.
  • [26] Portnoy, S., Asymptotic behavior of M-estimators of p regression parameters when $p^2/n$ is large. I. Consistency., Ann. Stat., 12 :1298–1309, 1984.
  • [27] Portnoy, S., Asymptotic behavior of the empiric distribution of M-estimated residuals from a regression model with many parameters., Ann. Stat., 14 :1152–1170, 1986.
  • [28] Shen, J., Shi, X., Sieve likelihood ratio inference on general parameter space., Science in China, 48(1):67–78, 2005.
  • [29] Spokoiny, V., Parametric estimation. Finite sample theory., Ann. Statist., 40(6) :2877–2909, 2012.
  • [30] Spokoiny, V., Bernstein–von Mises theorem for growing parameter dimension. Manuscript. ArXiv :1302.3430, 2013.
  • [31] Spokoiny, V., Wang, W., and Härdle, W., Local quantile regression (with rejoinder)., J. of Statistical Planing and Inference, 143(7) :1109–1129, 2013. ArXiv:1208.5384.
  • [32] van der Vaart, A. W. and Wellner, J., Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer, 1996.
  • [33] Wilks, S. S., The large-sample distribution of the likelihood ratio for testing composite hypotheses., Ann. Math. Statist., 9(1):60–62, 1938.
  • [34] Zaitsev, A., Burnaev, E., and Spokoiny, V., Properties of the posterior distribution of a regression model based on gaussian random fields., Automation and Remote Control, 74(10) :1645–1655, 2013.