The paper focuses on general properties of parametric minimum contrast estimators. The quality of estimation is measured in terms of the rate function related to the contrast, thus allowing to derive exponential risk bounds invariant with respect to the detailed probabilistic structure of the model. This approach works well for small or moderate samples and covers the case of a misspecified parametric model. Another important feature of the presented bounds is that they may be used in the case when the parametric set is not compact. These bounds do not rely on the entropy or covering numbers and can be easily computed. The most important statistical fact resulting from the exponential bonds is a concentration inequality which claims that minimum contrast estimators concentrate with a large probability on the level set of the rate function. In typical situations, every such set is a root-n neighborhood of the parameter of interest. We also show that the obtained bounds can help for bounding the estimation risk and constructing confidence sets for the underlying parameters. Our general results are illustrated for the case of an i.i.d. sample. We also consider several popular examples including least absolute deviation estimation and the problem of estimating the location of a change point. What we obtain in these examples slightly differs from the usual asymptotic results presented in statistical literature. This difference is due to the unboundness of the parameter set and a possible model misspecification.
References
Bahadur, R. (1960). On the asymptotic efficiency of tests and estimates. Sankhya.
Birgé, L. (2006). Model selection via testing: an alternative to (penalized) maximum likelihood estimators. Annales de l’Institut Henri Poincare (B) Probability and Statistics.
Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Relat. Fields, 97(1-2):113–150.
Birgé, L. and Massart, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli, 4(3):329–375.
Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Stat., 23:493–507.
Csorgő, M. and Horváth, L. (1997). Limit theorems in change-point analysis. Chichester: John Wiley & Sons.
Field, C. (1982). Small sample asymptotic expansions for multivariate M-estimates. Ann. Statist., 10(3):672–689.
Field, C. and Ronchetti, E. (1990). Small sample asymptotics. Institute of Mathematical Statistics Lecture Notes—Monograph Series, 13. Institute of Mathematical Statistics, Hayward, CA.
Huber, P. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. 5th Berkeley Symp. Math. Stat. Probab., Univ. Calif. 1965/66, 1, 221-233 (1967).
Huber, P. J. (1981). Robust statistics. John Wiley & Sons Inc., New York. Wiley Series in Probability and Mathematical Statistics.
Ibragimov, I. and Khas’minskij, R. (1981). Statistical estimation. Asymptotic theory. Transl. from the Russian by Samuel Kotz. New York - Heidelberg - Berlin: Springer-Verlag.
Jensen, J. L. and Wood, A. T. (1998). Large deviation and other results for minimum contrast estimators. Ann. Inst. Stat. Math., 50(4):673–695.
Koenker, R. (2005). Quantile regression. Cambridge University Press.
Koenker, R. and Xiao, Z. (2006). Quantile autoregression. J. Am. Stat. Assoc., 101(475):980–990.
Sieders, A. and Dzhaparidze, K. (1987). A large deviation result for parameter estimators and its application to nonlinear regression analysis. Ann. Stat., 15(3):1031–1049.
Van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat., 21(1):14–44.
Van der Vaart, A. and Wellner, J. A. (1996). Weak convergence and empirical processes. With applications to statistics. Springer Series in Statistics. New York, Springer.
