Explicit solution to the minimization problem of generalized cross-validation criterion for selecting ridge parameters in generalized ridge regression

Abstract

This paper considers optimization of the ridge parameters in generalized ridge regression (GRR) by minimizing a model selection criterion. GRR has a major advantage over ridge regression (RR) in that a solution to the minimization problem for one model selection criterion, i.e., Mallows’ $C_p$ criterion, can be obtained explicitly with GRR, but such a solution for any model selection criteria, e.g., $C_p$ criterion, cross-validation (CV) criterion, or generalized CV (GCV) criterion, cannot be obtained explicitly with RR. On the other hand, $C_p$ criterion is at a disadvantage compared to CV and GCV criteria because a good estimate of the error variance is required in order for $C_p$ criterion to work well. In this paper, we show that ridge parameters optimized by minimizing GCV criterion can also be obtained by closed forms in GRR. We can overcome one disadvantage of GRR by using GCV criterion for the optimization of ridge parameters. By using the result, we propose a principle component regression hybridized with the GRR that is a new method for a linear regression with highdimensional explanatory variables.