Annals of Statistics

A loss function approach to model specification testing and its relative efficiency

Yongmiao Hong and Yoon-Jin Lee

Full-text: Open access


The generalized likelihood ratio (GLR) test proposed by Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153–193] and Fan and Yao [Nonlinear Time Series: Nonparametric and Parametric Methods (2003) Springer] is a generally applicable nonparametric inference procedure. In this paper, we show that although it inherits many advantages of the parametric maximum likelihood ratio (LR) test, the GLR test does not have the optimal power property. We propose a generally applicable test based on loss functions, which measure discrepancies between the null and nonparametric alternative models and are more relevant to decision-making under uncertainty. The new test is asymptotically more powerful than the GLR test in terms of Pitman’s efficiency criterion. This efficiency gain holds no matter what smoothing parameter and kernel function are used and even when the true likelihood function is available for the GLR test.

Article information

Ann. Statist., Volume 41, Number 3 (2013), 1166-1203.

First available in Project Euclid: 13 June 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G10: Hypothesis testing

Efficiency generalized likelihood ratio test loss function local alternative kernel Pitman efficiency smoothing parameter


Hong, Yongmiao; Lee, Yoon-Jin. A loss function approach to model specification testing and its relative efficiency. Ann. Statist. 41 (2013), no. 3, 1166--1203. doi:10.1214/13-AOS1099.

Export citation


  • Azzalini, A., Bowman, A. W. and Härdle, W. (1989). On the use of nonparametric regression for model checking. Biometrika 76 1–11.
  • Azzalini, A. and Bowman, A. (1990). A look at some data on the Old Faithful geyser. Appl. Statist. 39 357–365.
  • Azzalini, A. and Bowman, A. (1993). On the use of nonparametric regression for checking linear relationships. J. Roy. Statist. Soc. Ser. B 55 549–557.
  • Bahadur, R. R. (1958). Examples of inconsistency of maximum likelihood estimates. Sankhyā 20 207–210.
  • Cai, Z., Fan, J. and Yao, Q. (2000). Functional-coefficient regression models for nonlinear time series. J. Amer. Statist. Assoc. 95 941–956.
  • Christoffersen, P. F. and Diebold, F. X. (1997). Optimal prediction under asymmetric loss. Econometric Theory 13 808–817.
  • Doukhan, P. (1994). Mixing: Properties and Examples. Lecture Notes in Statistics 85. Springer, New York.
  • Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica 50 987–1007.
  • Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11 1031–1057.
  • Fan, J. and Jiang, J. (2005). Nonparametric inferences for additive models. J. Amer. Statist. Assoc. 100 890–907.
  • Fan, J. and Jiang, J. (2007). Nonparametric inference with generalized likelihood ratio tests. TEST 16 409–444.
  • Fan, Y. and Li, Q. (2002). A consistent model specification test based on the kernel sum of squares of residuals. Econometric Rev. 21 337–352.
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
  • Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29 153–193.
  • Fan, J. and Zhang, C. (2003). A reexamination of diffusion estimators with applications to financial model validation. J. Amer. Statist. Assoc. 98 118–134.
  • Fan, J. and Zhang, W. (2004). Generalised likelihood ratio tests for spectral density. Biometrika 91 195–209.
  • Franke, J., Kreiss, J.-P. and Mammen, E. (2002). Bootstrap of kernel smoothing in nonlinear time series. Bernoulli 8 1–37.
  • Gao, J. and Gijbels, I. (2008). Bandwidth selection in nonparametric kernel testing. J. Amer. Statist. Assoc. 103 1584–1594.
  • Giacomini, R. and White, H. (2006). Tests of conditional predictive ability. Econometrica 74 1545–1578.
  • Granger, C. W. J. (1999). Outline of forecast theory using generalized cost functions. Spanish Economic Review 1 161–173.
  • Granger, C. W. J. and Pesaran, M. H. (1999). Economic and statistical measures of forecast accuracy. Cambridge Working Papers in Economics 9910, Faculty of Economics, Univ. Cambridge.
  • Granger, C. W. J. and Pesaran, M. H. (2000). Economic and statistical measures of forecast accuracy. Journal of Forecasting 19 537–560.
  • Granger, C. W. J. and Teräsvirta, T. (1993). Modelling Nonlinear Economic Relationships. Oxford Univ. Press, New York.
  • Hansen, B. (1999). Testing for linearity. Journal of Economic Survey 13 551–576.
  • Härdle, W. (1990). Applied Nonparametric Regression. Econometric Society Monographs 19. Cambridge Univ. Press, Cambridge.
  • Härdle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 1926–1947.
  • Hjellvik, V. and Tjøstheim, D. (1996). Nonparametric statistics for testing of linearity and serial independence. J. Nonparametr. Stat. 6 223–251.
  • Hong, Y. and Lee, Y.-J. (2005). Generalized spectral tests for conditional mean models in time series with conditional heteroscedasticity of unknown form. Rev. Econom. Stud. 72 499–541.
  • Hong, Y. and Lee, Y. (2013). Supplement to “A loss function approach to model specification testing and its relative efficiency.” DOI:10.1214/13-AOS1099SUPP.
  • Hong, Y. and White, H. (1995). Consistent specification testing via nonparametric series regression. Econometrica 63 1133–1159.
  • Horowitz, J. L. and Spokoiny, V. G. (2001). An adaptive, rate-optimal test of a parametric mean-regression model against a nonparametric alternative. Econometrica 69 599–631.
  • Ingster, Y. I. (1993a). Asymptotically minimax hypothesis testing for nonparametric alternatives. I. Math. Methods Statist. 2 85–114.
  • Ingster, Y. I. (1993b). Asymptotically minimax hypothesis testing for nonparametric alternatives. II. Math. Methods Statist. 3 1715–189.
  • Ingster, Y. I. (1993c). Asymptotically minimax hypothesis testing for nonparametric alternatives. III. Math. Methods Statist. 4 249–268.
  • Le Cam, L. (1990). Maximum likelihood—An introduction. ISI Review 58 153–171.
  • Lee, T.-H., White, H. and Granger, C. W. J. (1993). Testing for neglected nonlinearity in time series models: A comparison of neural network methods and alternative tests. J. Econometrics 56 269–290.
  • Lepski, O. V. and Spokoiny, V. G. (1999). Minimax nonparametric hypothesis testing: The case of an inhomogeneous alternative. Bernoulli 5 333–358.
  • Li, Q. and Racine, J. S. (2007). Nonparametric Econometrics: Theory and Practice. Princeton Univ. Press, Princeton, NJ.
  • Pagan, A. and Ullah, A. (1999). Nonparametric Econometrics. Cambridge Univ. Press, Cambridge.
  • Pan, J., Wang, H. and Yao, Q. (2007). Weighted least absolute deviations estimation for ARMA models with infinite variance. Econometric Theory 23 852–879.
  • Peel, D. A. and Nobay, A. R. (1998). Optimal monetary policy in a model of asymmetric central bank preferences. FMG Discussion Paper 0306.
  • Pesaran, M. H. and Skouras, S. (2001). Decision based methods for forecast evaluation. In Companion to Economic Forecasting (M. P. Clements and D. F. Hendry, eds.). Blackwell, Oxford.
  • Phillips, P. C. B. (1996). Econometric model determination. Econometrica 64 763–812.
  • Pitman, E. J. G. (1979). Some Basic Theory for Statistical Inference. Chapman & Hall, London.
  • Robinson, P. M. (1991). Consistent nonparametric entropy-based testing. Rev. Econom. Stud. 58 437–453.
  • Serfling, R. J. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
  • Sun, Y., Phillips, P. C. B. and Jin, S. (2008). Optimal bandwidth selection in heteroskedasticity–autocorrelation robust testing. Econometrica 76 175–194.
  • Tenreiro, C. (1997). Loi asymptotique des erreurs quadratiques intégrées des estimateurs à noyau de la densité et de la régression sous des conditions de dépendance. Port. Math. 54 187–213.
  • Varian, H. (1975). A Bayesian approach to real estate assessment. In Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage (S. E. Fienberg and A. Zellner, eds.). North-Holland, Amsterdam.
  • Vuong, Q. H. (1989). Likelihood ratio tests for model selection and nonnested hypotheses. Econometrica 57 307–333.
  • Weiss, A. A. (1996). Estimating time series models using the relevant cost function. J. Appl. Econometrics 11 539–560.
  • White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica 50 1–25.
  • Zellner, A. (1986). Bayesian estimation and prediction using asymmetric loss functions. J. Amer. Statist. Assoc. 81 446–451.
  • Zhang, C. and Dette, H. (2004). A power comparison between nonparametric regression tests. Statist. Probab. Lett. 66 289–301.

Supplemental materials

  • Supplementary material: Supplementary material for a loss function approach to model specification testing and its relative efficiency. In this supplement, we present the detailed proofs of Theorems 1–4 and report the simulation results with the bandwidth $h=S_{X}n^{-1/5}$.