Hiroshima Mathematical Journal

Selection of model selection criteria for multivariate ridge regression

Isamu Nagai

Full-text: Open access

Abstract

In the present study, we consider the selection of model selection criteria for multivariate ridge regression. There are several model selection criteria for selecting the ridge parameter in multivariate ridge regression, e.g., the $C_p$ criterion and the modified $C_p$ ($MC_p$) criterion. We propose the generalized $C_p$ ($GC_p$) criterion, which includes $C_p$ and $MC_p$ criteria as special cases. The $GC_p$ criterion is specified by a non-negative parameter $\lambda$, which is referred to as the penalty parameter. We attempt to select an optimal penalty parameter such that the predicted mean square error (PMSE) of the predictor of ridge regression after optimizing the ridge parameter is minimized. Through numerical experiments, we verify that the proposed optimization methods exhibit better performance than conventional optimization methods, i.e., optimizing only the ridge parameter by minimizing the $C_p$ or $MC_p$ criterion.

Article information

Source
Hiroshima Math. J., Volume 43, Number 1 (2013), 73-106.

Dates
First available in Project Euclid: 10 May 2013

Permanent link to this document
https://projecteuclid.org/euclid.hmj/1368217951

Digital Object Identifier
doi:10.32917/hmj/1368217951

Mathematical Reviews number (MathSciNet)
MR3066526

Zentralblatt MATH identifier
1348.62207

Subjects
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62H12: Estimation

Keywords
Asymptotic expansion generalized $C_p$ criterion model selection criterion multivariate linear regression model ridge regression selection of the model selection criterion

Citation

Nagai, Isamu. Selection of model selection criteria for multivariate ridge regression. Hiroshima Math. J. 43 (2013), no. 1, 73--106. doi:10.32917/hmj/1368217951. https://projecteuclid.org/euclid.hmj/1368217951


Export citation

References

  • A. C. Atkinson, A note on the generalized information criterion for choice of a model, Biometrika, 67 (1980), 413-418.
  • P. J. Brown and J. V. Zidek, Adaptive multivariate ridge regression, Ann. Statist., 8 (1980), 64–74.
  • B. Efron, The estimation of prediction error: covariance penalties and cross-validation, J. Amer. Statist. Assoc., 99 (2004), 619–632.
  • S. J. V. Dien, S. Iwatani, Y. Usuda and K. Matsui, Theoretical analysis of amino acid-producing Eschenrichia coli using a stoixhiometrix model and multivariate linear regression, J. Biosci. Bioeng., 102 (2006), 34–40.
  • Y. Fujikoshi and K. Satoh, Modified AIC and $C_p$ in multivariate linear regression, Biometrika, 84 (1997), 707–716.
  • Y. Fujikoshi, H. Yanagihara and H. Wakaki, Bias corrections of some criteria for selecting multivariate linear models in a general nonnormal case, Amer. J. Math. Management Sci., 25 (2005), 221-258.
  • Y. Haitovsky, On multivariate ridge regression, Biometrika, 74 (1987), 563–570.
  • A. E. Hoerl and R. W. Kennard, Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12(1970), 55–67.
  • J. C. Lagarias, J. A. Reeds, M. H. Wright and P. E. Wright, Convergence properties of the Nelder-Mead simplex method in low dimensions, SIAM J. Optim., 9 (1998), 112–147.
  • J. F. Lawless, Mean squared error properties of generalized ridge estimators, J. Amer. Statist. Assoc., 76 (1981), 462–466.
  • C. L. Mallows, Some comments on $C_p$, Technometrics, 15 (1973), 661–675.
  • C. L. Mallows, More comments on $C_p$, Technometrics, 37 (1995), 362–372.
  • C. S$\hat{\rm a}$rbu, C. Onisor, M. Posa, S. Kevresan and K. Kuhajda, Modeling and prediction (correction) of partition coefficients of bile acids and their derivatives by multivariate regression methods, Talanta, 75 (2008), 651–657.
  • R. Sax$\acute{\rm e}$n and J. Sundell, $^{137}$Cs in freshwater fish in Finland since 1986– a statistical analysis with multivariate linear regression models, J. Environ. Radioactiv., 87 (2006), 62–76.
  • X. Shen and J. Ye, Adaptive model selection, J. Amer. Statist. Assoc., 97 (2002), 210–221.
  • M. Siotani, T. Hayakawa and Y. Fujikoshi, Modern Multivariate Statistical Analysis: A Graduate Course and Handbook, American Sciences Press, Columbus, Ohio, 1985.
  • B. Skagerberg, J. MacGregor and C. Kiparissides, Multivariate data analysis applied to low-density polyethylene reactors, Chemometr. Intell. Lab. Syst., 14 (1992), 341–356.
  • R. S. Sparks, D. Coutsourides and L. Troskie, The multivariate $C_p$, Comm. Statist. A – Theory Methods, 12 (1983), 1775–1793.
  • M. S. Srivastava, Methods of Multivariate Statistics, John Wiley & Sons, New York, 2002.
  • N. H. Timm, Applied Multivariate Analysis, Springer-Verlag, New York, 2002.
  • H. Yanagihara and K. Satoh, An unbiased $C_p$ criterion for multivariate ridge regression, J. Multivariate Anal., 101 (2010), 1226–1238.
  • J. Ye, On measuring and correcting the effects of data mining and model selection, J. Amer. Statist. Assoc., 93 (1998), 120–131.
  • A. Yoshimoto, H. Yanagihara and Y. Ninomiya, Finding factors affecting a forest stand growth through multivariate linear modeling, J. Jpn. For. Res., 87 (2005), 504–512 (in Japanese).