The Annals of Statistics

Penalized Discriminant Analysis

Trevor Hastie, Andreas Buja, and Robert Tibshirani

Full-text: Open access

Abstract

Fisher's linear discriminant analysis (LDA) is a popular data-analytic tool for studying the relationship between a set of predictors and a categorical response. In this paper we describe a penalized version of LDA. It is designed for situations in which there are many highly correlated predictors, such as those obtained by discretizing a function, or the grey-scale values of the pixels in a series of images. In cases such as these it is natural, efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients, both for improved prediction performance and interpretability. We cast the classification problem into a regression framework via optimal scoring. Using this, our proposal facilitates the use of any penalized regression technique in the classification setting. The technique is illustrated with examples in speech recognition and handwritten character recognition.

Article information

Source
Ann. Statist. Volume 23, Number 1 (1995), 73-102.

Dates
First available: 11 April 2007

Permanent link to this document
http://projecteuclid.org/euclid.aos/1176324456

JSTOR
links.jstor.org

Digital Object Identifier
doi:10.1214/aos/1176324456

Mathematical Reviews number (MathSciNet)
MR1331657

Zentralblatt MATH identifier
0821.62031

Subjects
Primary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]
Secondary: 62G07: Density estimation

Keywords
Signal and image classification discrimination regularization

Citation

Hastie, Trevor; Buja, Andreas; Tibshirani, Robert. Penalized Discriminant Analysis. The Annals of Statistics 23 (1995), no. 1, 73--102. doi:10.1214/aos/1176324456. http://projecteuclid.org/euclid.aos/1176324456.


Export citation