Electronic Journal of Statistics

The bias and skewness of M-estimators in regression

Christopher Withers and Saralees Nadarajah

Full-text: Open access

Abstract

We consider M estimation of a regression model with a nuisance parameter and a vector of other parameters. The unknown distribution of the residuals is not assumed to be normal or symmetric. Simple and easily estimated formulas are given for the dominant terms of the bias and skewness of the parameter estimates. For the linear model these are proportional to the skewness of the ‘independent’ variables. For a nonlinear model, its linear component plays the role of these independent variables, and a second term must be added proportional to the covariance of its linear and quadratic components. For the least squares estimate with normal errors this term was derived by Box [1]. We also consider the effect of a large number of parameters, and the case of random independent variables.

Article information

Source
Electron. J. Statist., Volume 4 (2010), 1-14.

Dates
First available in Project Euclid: 7 January 2010

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1262876992

Digital Object Identifier
doi:10.1214/09-EJS447

Mathematical Reviews number (MathSciNet)
MR2579551

Zentralblatt MATH identifier
1329.62206

Subjects
Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Keywords
Bias reduction M-estimates non-linear regression robust skewness

Citation

Withers, Christopher; Nadarajah, Saralees. The bias and skewness of M -estimators in regression. Electron. J. Statist. 4 (2010), 1--14. doi:10.1214/09-EJS447. https://projecteuclid.org/euclid.ejs/1262876992


Export citation

References

  • [1] Box, M. J. (1971). Bias in non-linear estimation (with discussion)., Journal of the Royal Statistical Society B 33 171–201.
  • [2] Clarke, G. P. Y. (1980). Moments of the least squares estimators in a nonlinear regression model., Journal of the Royal Statistical Society B 42 227–237.
  • [3] Easton, G. S. and Ronchetti, E. (1986). General saddelpoint approximations with applications to, L statistics. Journal of the American Statistical Association 81 420–430.
  • [4] Hajek, J. and Sidak, Z. (1967)., Theory of Rank Tests. Academic Press, New York.
  • [5] Huber, P. J. (1981)., Robust statisitics. Wiley, New York.
  • [6] Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares indent estimators., Annals of Mathematical Statistics 40 633–643.
  • [7] Maronna, R. A., Martin, R. D. and Yohai, V. J. (2006)., Robust Statistics: Theory and Methods. Wiley, Chichester.
  • [8] Portnoy, S. (1984). Asymptotic behaviour of, M-estimators of p regression parameters when p2/n is large. I. Consistency. Annals of Statistics 12 1298–1309.
  • [9] Withers, C. S. (1982a). The distribution and quantiles of a function of parameter estimates., Annals of the Institute of Statistical Mathematics A 34 55–68.
  • [10] Withers, C. S. (1982b). Second order inference for asymptotically normal random variables., Sankhyā 44 19–27.
  • [11] Withers, C. S. (1983). Expansions for the distribution and quantiles of a regular functional of the empirical distribution with applications to nonparametric confidence intervals., Annals of Statistics 11 577–587.
  • [12] Withers, C. S. (1987). Bias reduction by Taylor series., Communications in Statistics—Theory and Methods 16 2369–2384.
  • [13] Withers, C. S. and Nadarajah, S. (2009). Asymptotic properties of, M-estimates. Technical Report, Applied Mathematics Group, Industrial Research Ltd., Lower Hutt, New Zealand.