Hiroshima Mathematical Journal

A class of cross-validatory model selection criteria

Hirokazu Yanagihara, Ke-Hai Yuan, Hironori Fujisawa, and Kentaro Hayashi

Full-text: Open access


In this paper, we define a class of cross-validatory model selection criteria as an estimator of the predictive risk function based on a discrepancy between a candidate model and the true model. For a vector of unknown parameters, $n$ estimators are required for the definition of the class, where $n$ is the sample size. The $i$th estimator $(i=1,\dots,n)$ is obtained by minimizing a weighted discrepancy function in which the $i$th observation has a weight of $1-\lambda$ and others have weight of $1$. Cross-validatory model selection criteria in the class are specified by the individual $\lambda$. The sample discrepancy function and the ordinary cross-validation (CV) criterion are special cases of the class. One may choose $\lambda$ to minimize the biases. The optimal $\lambda$ makes the bias-corrected CV (CCV) criterion a second-order unbiased estimator for the risk function, while the ordinary CV criterion is a first-order unbiased estimator of the risk function.

Article information

Hiroshima Math. J., Volume 43, Number 2 (2013), 149-177.

First available in Project Euclid: 25 June 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62H25: Factor analysis and principal components; correspondence analysis
Secondary: 62F07: Ranking and selection

Asymptotic expansion bias correction cross-validation criterion model misspecification model selection predictive discrepancy sample discrepancy function structural equation model


Yanagihara, Hirokazu; Yuan, Ke-Hai; Fujisawa, Hironori; Hayashi, Kentaro. A class of cross-validatory model selection criteria. Hiroshima Math. J. 43 (2013), no. 2, 149--177. doi:10.32917/hmj/1372180510. https://projecteuclid.org/euclid.hmj/1372180510

Export citation


  • H. Akaike, Information theory and an extension of the maximum likelihood principle, 2nd. International Symposium on Information Theory (eds. B. N. Petrov and F. Csáki), 267–281, Akadémiai Kiadó, Budapest, 1973. \labelAkaike1973
  • H. Akaike, A new look at the statistical model identification, IEEE Trans. Automatic Control AC-19 (1974), 716–723. \labelAkaike1974
  • H. Akaike, Factor analysis and AIC, Psychometrika, 52 (1987), 317–332.
  • N. Balakrishnan and R. S. Ambagaspitiya, On skew Laplace distribution, Technical Report, Department of Mathematics & Statistics, McMaster University, Hamilton, Ontario, Canada, 1994.
  • D. L. Bandalos, Factors influencing cross-validation of confirmatory factor analysis models, Mult. Behavioral Res., 28 (1993), 351–342.
  • A. Basu, I. R. Harris, N. L. Hjort and M. C. Jones, Robust and efficient estimation by minimising a density power divergence, Biometrika, 85 (1998), 549–559.
  • H. Bozdogan, Model selection and Akaike's information criterion (AIC): the general theory and its analytical extensions, Psychometrika, 52 (1987), 345–370.
  • M. W. Browne and R. Cudeck, Single sample cross-validation indices for covariance structures, Mult. Behavioral Res., 24 (1989), 445–455.
  • R. Cudeck and M. W. Browne, Cross-validation of covariance structures, Mult. Behavioral Res., 18 (1983), 147–167.
  • J. G. De Gooijer, Cross-validation criteria for covariance structures, Comm. Statist. Simulation Comput., 24 (1995), 111–147.
  • Y. Fujikoshi, T. Noguchi, M. Ohtaki and H. Yanagihara, Corrected versions of cross-validation criteria for selecting multivariate regression and growth curve models, Ann. Inst. Statist. Math., 55 (2003), 537–553.
  • H. Fujisawa and S. Eguchi, Robust estimation in the normal mixture model, J. Statist. Plann. Inference, 136 (2006), 3989–4011.
  • M. Ichikawa and S. Konishi, Model evaluation and information criteria in covariance structure analysis, British J. Math. Statist. Psych., 52 (1999), 285–302.
  • M. Ishiguro, Y. Sakamoto and G. Kitagawa, Bootstrapping log likelihood and EIC, an extension of AIC, Ann. Inst. Statist. Math., 49 (1997), 411–434.
  • S. Konishi, Statistical model evaluation and information criteria, In Multivariate Analysis, Design of Experiments, and Survey Sampling (ed. S. Ghosh), 369–399, Marcel Dekker, New York, 1999.
  • S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Statistics, 22 (1951), 79–86.
  • S.-Y. Lee and E. J. Kontoghiorghes (eds.), Handbook of Latent Variable and Related Models, Elsevier, Amsterdam, 2007.
  • E. L. Lehmann and G. Casella, Theory of Point Estimation (2nd. edition), Springer-Verlag, New York, 1998. \labelLehmann1998
  • J. R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics (revised ed.), John Wiley & Sons, New York, 1999. \labelMagnus1999
  • K. V. Mardia, Measures of multivariate skewness and kurtosis with applications, Biometrika, 57 (1970), 519–530.
  • G. Schwarz, Estimating the dimension of a model, Ann. Statist., 6 (1978), 461–464.
  • M. Stone, Cross-validatory choice and assessment of statistical predictions, J. Roy. Statist. Soc. Ser. B, 36 (1974), 111–147. \labelStone1974
  • M. Stone, An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion, J. Roy. Statist. Soc. Ser. B, 39 (1977), 44–47. \labelStone1977
  • K. Takeuchi, Distribution of information statistics and criteria for adequacy of models, Math. Sci., 153 (1976), 12–18 (in Japanese).
  • H. White, Maximum likelihood estimation of misspecified models, Econometrica, 50 (1982), 1–25.
  • H. Yanagihara, Selection of covariance structure models in nonnormal data by using information criterion: an application to data from the survey of the Japanese national character, Proc. Inst. Statist. Math., 53 (2005), 133–157 (in Japanese).
  • H. Yanagihara, A family of estimators for multivariate kurtosis in a nonnormal linear regression model, J. Multivariate Anal., 98, 1–29.
  • H. Yanagihara and K.-H. Yuan, Four improved statistics for contrasting means by correcting skewness and kurtosis, British J. Math. Statist. Psych., 58 (2005), 209–237.
  • H. Yanagihara, T. Tonda and C. Matsumoto, Bias correction of cross-validation criterion based on Kullback-Leibler information under a general condition, J. Multivariate Anal., 97 (2006), 1965–1975.
  • H. Yanagihara, K. Kamo and T. Tonda, Second-order bias-corrected AIC in multivariate normal linear models under nonnormality, Canad. J. Statist., 39 (2011), 126–146.
  • H. Yanagihara, T. Himeno and K.-H. Yuan, GLS discrepancy based information criteria for selecting covariance structure models, Behaviormetrika, 37 (2010), 71–86.
  • K.-H. Yuan and P. M. Bentler, Structural equation modeling, In Handbook of Statistics 26: Psychometrics (eds. C. R. Rao and S. Sinharay), 97–358, North-Holland, Amsterdam, 2007.