The Annals of Statistics

Empirical process of residuals for high-dimensional linear models

Enno Mammen

Full-text: Open access


We give a stochastic expansion for the empirical distribution function $\hat{F}_n$ of residuals in a p-dimensional linear model. This expansion holds for p increasing with n. It shows that, for high-dimensional linear models, $\hat{F}_n$ strongly depends on the chosen estimator $\hat{\theta}$ of the parameter $\theta$ of the linear model. In particular, if one uses an ML-estimator $\hat{\theta}_{ML}$ which is ML motivated by a wrongly specified error distribution function G, then $\hat{F}_n$ is biased toward G. For p^2 / n \to \infty$, this bias effect is of larger order than the stochastic fluctuations of the empirical process. Hence, the statistical analysis may just reproduce the assumptions imposed.

Article information

Ann. Statist., Volume 24, Number 1 (1996), 307-335.

First available in Project Euclid: 26 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G30: Order statistics; empirical distribution functions
Secondary: 62J05: Linear regression 62J20: Diagnostics

Empirical processes residuals linear models asymptotics with increasing dimension


Mammen, Enno. Empirical process of residuals for high-dimensional linear models. Ann. Statist. 24 (1996), no. 1, 307--335. doi:10.1214/aos/1033066211.

Export citation


  • BICKEL, P. J. and FREEDMAN, D. A. 1983. Bootstrapping regression models with many parameZ. ters. In A Festschrift for Erich Lehmann P. Bickel, K. Doksum and J. L. Hodges, eds. 28 48. Wadsworth, Belmont, CA. Z.
  • EHM, W. 1986. On maximum likelihood estimation in high-dimensional log-linear ty pe models. I. The independent case. Unpublished manuscript, Sonderforschungsbereich 123, Univ. Heidelberg. Z.
  • HABERMAN, S. J. 1977a. Log-linear and frequency tables with small expected cell counts. Ann. Statist. 5 1148 1169. Z.
  • HABERMAN, S. J. 1977b. Maximum likelihood estimates in exponential response models. Ann. Statist. 5 815 841. Z. HOEFFDING 1963. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13 30. Z.
  • HUBER, P. J. 1981. Robust Statistics. Wiley, New York. Z.
  • IOANNIDES, E. 1987. Asy mptotik des empirischen Prozesses der Residuen und Schatzen der ¨ Form der Fehlerverteilung im linearen Regressionsmodell. Diplomarbeit, Fakultat fur ¨ ¨ Mathematik, Univ. Heidelberg. Z.
  • KANTOROWITSCH, L. W. and AKILOW, G. P. 1964. Funktionalanalysis in Normierten Raumen. ¨ Akademie Verlag, Berlin. Z.
  • KOUL, H. 1969. Asy mptotic behavior of Wilcoxon ty pe confidence regions in multiple linear regression. Ann. Math. Statist. 40 1950 1979. Z.
  • KOUL, H. 1970. Some convergence theorems for ranks and weighted empirical cumulatives. Ann. Math. Statist. 41 1768 1773. Z.
  • KOUL, H. 1984. Tests of goodness-of-fit in linear regression. Colloq. Math. Soc. Janos Boly ai 45 ´ 279 315.
  • KOUL, H. 1992. Weighted Empiricals and Linear Models. IMS, Hay ward, CA. Z.
  • KREISS, J.-P. 1988. Asy mptotic statistical inference for a class of stochastic processes. Habilitationsschrift, Fachbereich Mathematik, Univ. Hamburg. Z.
  • KREISS, J.-P. 1991. Estimation of the distribution function of noise in stationary processes. Metrika 38 285 297. Z.
  • LOy NES, R. M. 1980. The empirical distribution function of residuals from generalised regression. Ann. Statist. 8 285 298. Z.
  • MAMMEN, E. 1989. Asy mptotics with increasing dimension for robust regression with applications to the bootstrap. Ann. Statist. 17 382 400. Z.
  • MAMMEN, E. 1993. Bootstrap and wild bootstrap for high-dimensional linear models. Ann. Statist. 21 255 285. Z.
  • MO, M. 1991. Asy mptotic normality of minimum contrast estimators. Unpublished manuscript, Dept. Statistics, Univ. Toronto. Z.
  • MO, M. 1992. Bootstrapping with increasing dimension. Unpublished manuscript, Dept. Statistics, Univ. Toronto. Z.
  • POLLARD, D. 1984. Convergence of Stochastic Processes. Springer, New York. Z.
  • PORTNOY, S. 1984. Asy mptotic behaviour of M-estimators of p regression parameters when p2 n is large. I. Consistency. Ann. Statist. 12 1298 1309. Z.
  • PORTNOY, S. 1985. Asy mptotic behaviour of M-estimator of p regression parameters when p2 n is large. II. Normal approximation. Ann. Statist. 13 1403 1417. Z.
  • PORTNOY, S. 1986. Asy mptotic behaviour of the empiric distribution of M-estimated residuals from a regression model with many parameters. Ann. Statist. 14 1152 1170. Z.
  • PORTNOY, S. 1988. Asy mptotic behaviour of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Statist. 16 356 366. Z.
  • SAUERMANN, W. 1989. Bootstrapping the maximum likelihood estimator in high-dimensional log-linear models. Ann. Statist. 17 1198 1216. Z.
  • SHORACK, G. R. 1982. Bootstrapping robust regression. Comm. Statist. Theory Methods 11 961 972. Z.
  • SHORACK, G. R. and WELLNER, J. A. 1986. Empirical Processes with Applications to Statistics. Wiley, New York. Z.
  • WELSH, A. H. 1989. On M-processes and M-estimation Ann. Statist. 17 337 361. Z.
  • WHITTLE, P. 1960. Bounds for the moments of linear and quadratic forms in independent variables. Theory Probab. Appl. 5 302 305.