Orthogonalized smoothing for rescaled spike and slab models
Hemant Ishwaran, Ariadni Papana
Abstract
Rescaled spike and slab models are a new Bayesian variable selection method for linear regression models. In high dimensional orthogonal settings such models have been shown to possess optimal model selection properties. We review background theory and discuss applications of rescaled spike and slab models to prediction problems involving orthogonal polynomials. We first consider global smoothing and discuss potential weaknesses. Some of these deficiencies are remedied by using local regression. The local regression approach relies on an intimate connection between local weighted regression and weighted generalized ridge regression. An important implication is that one can trace the effective degrees of freedom of a curve as a way to visualize and classify curvature. Several motivating examples are presented.
First Page:
Show
Hide
Primary Subjects: 62J07
Secondary Subjects: 62J05
Keywords: effective degrees of freedom; penalization; selective shrinkage
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.imsc/1209398474
Digital Object Identifier: doi:10.1214/074921708000000192
References
[1] Ishwaran, H. and Rao, J. S. (2005). Spike and slab variable selection: frequentist and Bayesian strategies. Ann. Statist. 33 730–773.
Mathematical Reviews (MathSciNet):
MR2163158
Zentralblatt MATH:
1068.62079
Digital Object Identifier: doi:10.1214/009053604000001147
Project Euclid: euclid.aos/1117114335
[2] Ishwaran, H. and Rao, J. S. (2005). Spike and slab gene selection for multigroup microarray data. J. Amer. Statist. Assoc. 100 764–780.
Mathematical Reviews (MathSciNet):
MR2201009
Zentralblatt MATH:
1117.62363
Digital Object Identifier: doi:10.1198/016214505000000051
[3] Ishwaran, H. and Rao, J. S. (2003). Detecting differentially expressed genes in microarrays using Bayesian model selection. J. Amer. Statist. Assoc. 98 438–455.
Mathematical Reviews (MathSciNet):
MR1995720
Zentralblatt MATH:
1041.62090
Digital Object Identifier: doi:10.1198/016214503000224
[4] Lempers, F. B. (1971). Posterior Probabilities of Alternative Linear Models. Rotterdam Univ. Press, Rotterdam.
Mathematical Reviews (MathSciNet):
MR359234
Zentralblatt MATH:
0246.62010
[5] Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. J. Amer. Statist. Assoc. 83 1023–1036.
Mathematical Reviews (MathSciNet):
MR997578
Zentralblatt MATH:
0673.62051
Digital Object Identifier: doi:10.2307/2290129
JSTOR: links.jstor.org
[6] George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881–889.
[7] George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339–373.
[8] Chipman, H. (1996). Bayesian variable selection with related predictors. Canad. J. Statist. 24 17–36.
Mathematical Reviews (MathSciNet):
MR1394738
Digital Object Identifier: doi:10.2307/3315687
JSTOR: links.jstor.org
[9] Clyde, M., DeSimone, H. and Parmigiani, G. (1996). Prediction via orthogonalized model mixing. J. Amer. Statist. Assoc. 91 1197–1208.
[10] Geweke, J. and Meese, R. (1981). Estimating regression models of finite but unknown order. International Econ. Rev. 22 55–70.
Mathematical Reviews (MathSciNet):
MR614347
Zentralblatt MATH:
0457.62050
Digital Object Identifier: doi:10.2307/2526135
JSTOR: links.jstor.org
[11] Chipman, H. A., George, E. I. and McCulloch R. E. (2001). The practical implementation of Bayesian model selection (with discussion). In IMS Lecture Notes Monogr. Ser. 38. Model Selection (P. Lahiri, ed.) 63–134. IMS, Beachwood, OH.
Mathematical Reviews (MathSciNet):
MR2000752
Digital Object Identifier: doi:10.1214/lnms/1215540964
[12] Bachrach, L. K., Hastie, T., Wang, M.-C. and Narasimhan, B. (1999). Bone mineral acquisition in healthy asian, hispanic, black and caucasian youth. A longitudinal study. J. Clin. Endocrinol. Metab. 84 4702–4712.
[13] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
Mathematical Reviews (MathSciNet):
MR1082147
[14] Hastie, T. and Loader, C. (1993). Local regression: automatic kernel carpentry. Statist. Sci. 8 120–143.
[15] Stone, C. J. (1977). Consistent nonparametric regression. Ann. Statist. 5 595–620.
Mathematical Reviews (MathSciNet):
MR443204
Zentralblatt MATH:
0366.62051
Digital Object Identifier: doi:10.1214/aos/1176343886
Project Euclid: euclid.aos/1176343886
[16] Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829–836.
Mathematical Reviews (MathSciNet):
MR556476
Zentralblatt MATH:
0423.62029
Digital Object Identifier: doi:10.2307/2286407
[17] Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998–1004.
Mathematical Reviews (MathSciNet):
MR1209561
Zentralblatt MATH:
0850.62354
Digital Object Identifier: doi:10.2307/2290637
JSTOR: links.jstor.org
[18] Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57 371–394.
Mathematical Reviews (MathSciNet):
MR1323345
JSTOR: links.jstor.org
[19] Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215–1230.
Mathematical Reviews (MathSciNet):
MR760684
Zentralblatt MATH:
0554.62035
Digital Object Identifier: doi:10.1214/aos/1176346788
Project Euclid: euclid.aos/1176346788
[20] Genovese, C. R., Miller, C. J., Nichol, R. C., Arjunwadkar, M. and Wasserman, L. (2004). Nonparametric inference for the cosmic microwave background. Statist. Sci. 19 308–321.
Mathematical Reviews (MathSciNet):
MR2146946
Digital Object Identifier: doi:10.1214/088342304000000161
Project Euclid: euclid.ss/1105714165
[21] Li, X., Gentleman, R., Lu, X., Shi, Q., Iglehart, J. D., Harris, L. and Miron, A. (2005). SELDI-TOF mass spectometry protein data. In Bioinformatics and Computational Biology Solutions Using R and Bioconductor (R. Gentleman et al., eds.) 91–108. Springer, New York.
Institute of Mathematical Statistics Collections
