Model selection rates of information based criteria

Abstract

Model selection criteria proposed over the years have become common procedures in applied research. This article examines the true model selection rates of any model selection criteria; with true model meaning the data generating model. The rate at which model selection criteria select the true model is important because the decision of model selection criteria affects both interpretation and prediction.

This article provides a general functional form for the mean function of the true model selection rates process, for any model selection criteria. Until now, no other article has provided a general form for the mean function of true model selection rate processes. As an illustration of the general form, this article provides the mean function for the true model selection rates of two commonly used model selection criteria, Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC). The simulations reveal deeper insight into properties of consistency and efficiency of AIC and BIC. Furthermore, the methodology proposed here for tracking the mean function of model selection procedures, which is based on accuracy of selection, lends itself for determining sufficient sample size in linear models for reliable inference in model selection.