We consider M estimation of a regression model with a nuisance parameter and a vector of other parameters. The unknown distribution of the residuals is not assumed to be normal or symmetric. Simple and easily estimated formulas are given for the dominant terms of the bias and skewness of the parameter estimates. For the linear model these are proportional to the skewness of the ‘independent’ variables. For a nonlinear model, its linear component plays the role of these independent variables, and a second term must be added proportional to the covariance of its linear and quadratic components. For the least squares estimate with normal errors this term was derived by Box [1]. We also consider the effect of a large number of parameters, and the case of random independent variables.
References
[1] Box, M. J. (1971). Bias in non-linear estimation (with discussion)., Journal of the Royal Statistical Society B 33 171–201.
Mathematical Reviews (MathSciNet):
MR315827
[2] Clarke, G. P. Y. (1980). Moments of the least squares estimators in a nonlinear regression model., Journal of the Royal Statistical Society B 42 227–237.
Mathematical Reviews (MathSciNet):
MR583361
[3] Easton, G. S. and Ronchetti, E. (1986). General saddelpoint approximations with applications to, L statistics. Journal of the American Statistical Association 81 420–430.
Mathematical Reviews (MathSciNet):
MR845879
[4] Hajek, J. and Sidak, Z. (1967)., Theory of Rank Tests. Academic Press, New York.
Mathematical Reviews (MathSciNet):
MR229351
[5] Huber, P. J. (1981)., Robust statisitics. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR606374
[6] Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares indent estimators., Annals of Mathematical Statistics 40 633–643.
Mathematical Reviews (MathSciNet):
MR238419
[7] Maronna, R. A., Martin, R. D. and Yohai, V. J. (2006)., Robust Statistics: Theory and Methods. Wiley, Chichester.
[8] Portnoy, S. (1984). Asymptotic behaviour of, M-estimators of p regression parameters when p2/n is large. I. Consistency. Annals of Statistics 12 1298–1309.
Mathematical Reviews (MathSciNet):
MR760690
[9] Withers, C. S. (1982a). The distribution and quantiles of a function of parameter estimates., Annals of the Institute of Statistical Mathematics A 34 55–68.
Mathematical Reviews (MathSciNet):
MR650324
[10] Withers, C. S. (1982b). Second order inference for asymptotically normal random variables., Sankhyā 44 19–27.
Mathematical Reviews (MathSciNet):
MR692891
[11] Withers, C. S. (1983). Expansions for the distribution and quantiles of a regular functional of the empirical distribution with applications to nonparametric confidence intervals., Annals of Statistics 11 577–587.
Mathematical Reviews (MathSciNet):
MR696069
[12] Withers, C. S. (1987). Bias reduction by Taylor series., Communications in Statistics—Theory and Methods 16 2369–2384.
Mathematical Reviews (MathSciNet):
MR915469
[13] Withers, C. S. and Nadarajah, S. (2009). Asymptotic properties of, M-estimates. Technical Report, Applied Mathematics Group, Industrial Research Ltd., Lower Hutt, New Zealand.
