The Annals of Statistics

Minimax estimation of a functional on a structured high-dimensional model

James M. Robins, Lingling Li, Rajarshi Mukherjee, Eric Tchetgen Tchetgen, and Aad van der Vaart

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We introduce a new method of estimation of parameters in semiparametric and nonparametric models. The method employs $U$-statistics that are based on higher-order influence functions of the parameter of interest, which extend ordinary linear influence functions, and represent higher derivatives of this parameter. For parameters for which the representation cannot be perfect the method often leads to a bias-variance trade-off, and results in estimators that converge at a slower than $\sqrt{n}$-rate. In a number of examples, the resulting rate can be shown to be optimal. We are particularly interested in estimating parameters in models with a nuisance parameter of high dimension or low regularity, where the parameter of interest cannot be estimated at $\sqrt{n}$-rate, but we also consider efficient $\sqrt{n}$-estimation using novel nonlinear estimators. The general approach is applied in detail to the example of estimating a mean response when the response is not always observed.

Article information

Source
Ann. Statist., Volume 45, Number 5 (2017), 1951-1987.

Dates
Received: April 2015
Revised: August 2016
First available in Project Euclid: 31 October 2017

Permanent link to this document
https://projecteuclid.org/euclid.aos/1509436824

Digital Object Identifier
doi:10.1214/16-AOS1515

Mathematical Reviews number (MathSciNet)
MR3718158

Zentralblatt MATH identifier
06821115

Subjects
Primary: 62G05: Estimation 62G20: Asymptotic properties 62F25: Tolerance and confidence regions

Keywords
Nonlinear functional nonparametric estimation $U$-statistic influence function tangent space

Citation

Robins, James M.; Li, Lingling; Mukherjee, Rajarshi; Tchetgen, Eric Tchetgen; van der Vaart, Aad. Minimax estimation of a functional on a structured high-dimensional model. Ann. Statist. 45 (2017), no. 5, 1951--1987. doi:10.1214/16-AOS1515. https://projecteuclid.org/euclid.aos/1509436824


Export citation

References

  • [1] Bickel, P. J. (1982). On adaptive estimation. Ann. Statist. 10 647–671.
  • [2] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Univ. Press, Baltimore, MD.
  • [3] Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā Ser. A 50 381–393.
  • [4] Birgé, L. and Massart, P. (1995). Estimation of integral functionals of a density. Ann. Statist. 23 11–29.
  • [5] Bolthausen, E., Perkins, E. and van der Vaart, A. (2002). Lectures on Probability Theory and Statistics (P. Bernard, ed.). Lecture Notes in Math. 1781. Springer, Berlin.
  • [6] Cai, T. T. and Low, M. G. (2005). Nonquadratic estimators of a quadratic functional. Ann. Statist. 33 2930–2956.
  • [7] Cai, T. T. and Low, M. G. (2006). Optimal adaptive estimation of a quadratic functional. Ann. Statist. 34 2298–2325.
  • [8] Donoho, D. L. and Nussbaum, M. (1990). Minimax quadratic estimation of a quadratic functional. J. Complexity 6 290–323.
  • [9] Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley, New York.
  • [10] Has’minskiĭ, R. Z. and Ibragimov, I. A. (1979). On the nonparametric estimation of functionals. In Proceedings of the Second Prague Symposium on Asymptotic Statistics (Hradec Králové, 1978) 41–51. North-Holland, Amsterdam.
  • [11] Huber, P. J. and Ronchetti, E. M. (2009). Robust Statistics, 2nd ed. Wiley, Hoboken, NJ.
  • [12] Kerkyacharian, G. and Picard, D. (1996). Estimating nonquadratic functionals of a density using Haar wavelets. Ann. Statist. 24 485–507.
  • [13] Koševnik, J. A. and Levit, B. J. (1976). On a nonparametric analogue of the information matrix. Teor. Verojatnost. i Primenen. 21 759–774.
  • [14] Laurent, B. (1997). Estimation of integral functionals of a density and its derivatives. Bernoulli 3 181–211.
  • [15] Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 1302–1338.
  • [16] Lindsay, B. G. (1983). Efficiency of the conditional score in a mixture setting. Ann. Statist. 11 486–497.
  • [17] Murphy, S. A. and van der Vaart, A. W. (2000). On profile likelihood. J. Amer. Statist. Assoc. 95 449–485.
  • [18] Nemirovski, A. (2000). Topics in non-parametric statistics. In Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85–277. Springer, Berlin.
  • [19] Pfanzagl, J. (1982). Contributions to a General Asymptotic Statistical Theory. Lecture Notes in Statistics 13. Springer, New York.
  • [20] Pfanzagl, J. (1985). Asymptotic Expansions for General Statistical Models. Lecture Notes in Statistics 31. Springer, Berlin.
  • [21] Pfanzagl, J. (1990). Estimation in Semiparametric Models: Some Recent Developments. Lecture Notes in Statistics 63. Springer, New York.
  • [22] Robins, J. M., Li, L., Mukherjee, R., Tchetgen, E. T. and van der Vaart, A. (2017). Supplement to “Minimax estimation of a functional on a structured high-dimensional model.” DOI:10.1214/16-AOS1515SUPP.
  • [23] Robins, J., Li, L., Tchetgen, E. and van der Vaart, A. (2008). Higher order influence functions and minimax estimation of nonlinear functionals. In Probability and Statistics: Essays in Honor of David A. Freedman. Inst. Math. Stat. Collect. 2 335–421. IMS, Beachwood, OH.
  • [24] Robins, J., Li, L., Tchetgen, E. and van der Vaart, A. W. (2009). Quadratic semiparametric von Mises calculus. Metrika 69 227–247.
  • [25] Robins, J., Tchetgen Tchetgen, E., Li, L. and van der Vaart, A. (2009). Semiparametric minimax rates. Electron. J. Stat. 3 1305–1321.
  • [26] Robins, J. M. and Rotnitzky, A. (1995). Semiparametric efficiency in multivariate regression models with missing data. J. Amer. Statist. Assoc. 90 122–129.
  • [27] Rotnitzky, A. and Robins, J. M. (1995). Semi-parametric estimation of models for means and covariances in the presence of missing data. Scand. J. Statist. 22 323–333.
  • [28] Tsybakov, A. B. (2004). Introduction à l’estimation non-paramétrique. Mathématiques & Applications (Berlin) [Mathematics & Applications] 41. Springer, Berlin.
  • [29] van Zwet, W. R. (1984). A Berry–Esseen bound for symmetric statistics. Z. Wahrsch. Verw. Gebiete 66 425–440.
  • [30] van der Laan, M. J. and Robins, J. M. (2003). Unified Methods for Censored Longitudinal Data and Causality. Springer, New York.
  • [31] van der Vaart, A. (1991). On differentiable functionals. Ann. Statist. 19 178–204.
  • [32] Van der Vaart, A. (1996). Efficient maximum likelihood estimation in semiparametric mixture models. Ann. Statist. 24 862–878.
  • [33] van der Vaart, A. (2014). Higher order tangent spaces and influence functions. Statist. Sci. 29 679–686.
  • [34] van der Vaart, A. W. (1988). Estimating a real parameter in a class of semiparametric models. Ann. Statist. 16 1450–1474.
  • [35] van der Vaart, A. W. (1988). Statistical Estimation in Large Parameter Spaces. CWI Tract 44. Stichting Mathematisch Centrum, Centrum voor Wiskunde en Informatica, Amsterdam.
  • [36] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [37] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [38] von Mises, R. (1947). On the asymptotic distribution of differentiable statistical functions. Ann. Math. Stat. 18 309–348.
  • [39] Waterman, R. P. and Lindsay, B. G. (1996). Projected score methods for approximating conditional scores. Biometrika 83 1–13.

Supplemental materials

  • Supplement to “Minimax estimation of a functional on a structured high-dimensional model”. The remainder of the paper is given in the supplement.