The Annals of Statistics

On the degrees of freedom in shape-restricted regression

Mary Meyer and Michael Woodroofe

Full-text: Open access

Abstract

For the problem of estimating a regression function, $\mu$ say, subject to shape constraints, like monotonicity or convexity, it is argued that the divergence of the maximum likelihood estimator provides a useful measure of the effective dimension of the model. Inequalities are derived for the expected mean squared error of the maximum likelihood estimator and the expected residual sum of squares. These generalize equalities from the case of linear regression. As an application, it is shown that the maximum likelihood estimator of the error variance $\sigma^2$ is asymptotically normal with mean $\sigma^2$ and variance $2\sigma_2/n$. For monotone regression, it is shown that the maximum likelihood estimator of $\mu$ attains the optimal rate of convergence, and a bias correction to the maximum likelihood estimator of $\sigma^2$ is derived.

Article information

Source
Ann. Statist. Volume 28, Number 4 (2000), 1083-1104.

Dates
First available in Project Euclid: 12 March 2002

Permanent link to this document
http://projecteuclid.org/euclid.aos/1015956708

Digital Object Identifier
doi:10.1214/aos/1015956708

Mathematical Reviews number (MathSciNet)
MR1810920

Zentralblatt MATH identifier
1105.62340

Subjects
Primary: 62G08: Nonparametric regression

Keywords
Asymptotic distribution bias reduction divergence effective dimension simulation Stein's identity variance estimation

Citation

Meyer, Mary; Woodroofe, Michael. On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 (2000), no. 4, 1083--1104. doi:10.1214/aos/1015956708. http://projecteuclid.org/euclid.aos/1015956708.


Export citation

References

  • Breiman, L. (1968). Probability. Addison-Weseley, Reading, MA.
  • Donoho, D. (1990). Gelfand n-widths and the method of least squares. Unpublished manuscript.
  • Donoho, D. and Johnstone, I. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1225.
  • Efromovich, S. (1997). On quasi-linear wavelet estimation. J. Amer. Statist. Assoc. 94 189-204.
  • Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2. Wiley, New York.
  • Gasser, T. Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in non-linear regression. Biometrika 73 625-633.
  • Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. Le Cam and R. Olshen, eds.) 2 539-558. Wadsworth and Brooks/Cole, Belmont, CA.
  • Groeneboom, P. (1989). Brownian motion with a parabolic drift and airy functions. Probab. Theory Related Fields 81 79-109.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Klass, M. (1983). On the maximum of a random walkwith a small negative drift. Ann. Probab. 11 491-505.
  • Meyer, M. (1996). Shape restricted inference with applications to nonparametric regression, smooth nonparametric regression, and density estimation. Ph.D. thesis, Dept. statistics, Univ. Michigan.
  • Meyer, M. and Woodroofe, M. (1998). Variance estimation in monotone regression. Technical Report 322, Dept. Statistics, Univ. Michigan.
  • Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215-1230.
  • Robertson, T., Wright, F. and Dykstra, R. (1988). Order Restricted Inference, Wiley, New York.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151.
  • Van der Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924.
  • Woodroofe, M. (1982). Non-linear Renewal Theory in Sequential Analysis. SIAM, Philadelphia.
  • Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 449-453.