The Annals of Statistics

Consistent Nonparametric Regression

Charles J. Stone

Full-text: Open access

Abstract

Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$-valued and $Y$ is $\mathbb{R}^{d'}$-valued. Given a random sample $(X_1, Y_1), \cdots, (X_n, Y_n)$ from the distribution of $(X, Y)$, the conditional distribution $P^Y(\bullet \mid X)$ of $Y$ given $X$ can be estimated nonparametrically by $\hat{P}_n^Y(A \mid X) = \sum^n_1 W_{ni}(X)I_A(Y_i)$, where the weight function $W_n$ is of the form $W_{ni}(X) = W_{ni}(X, X_1, \cdots, X_n), 1 \leqq i \leqq n$. The weight function $W_n$ is called a probability weight function if it is nonnegative and $\sum^n_1 W_{ni}(X) = 1$. Associated with $\hat{P}_n^Y(\bullet \mid X)$ in a natural way are nonparametric estimators of conditional expectations, variances, covariances, standard deviations, correlations and quantiles and nonparametric approximate Bayes rules in prediction and multiple classification problems. Consistency of a sequence $\{W_n\}$ of weight functions is defined and sufficient conditions for consistency are obtained. When applied to sequences of probability weight functions, these conditions are both necessary and sufficient. Consistent sequences of probability weight functions defined in terms of nearest neighbors are constructed. The results are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.

Article information

Source
Ann. Statist., Volume 5, Number 4 (1977), 595-620.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176343886

Digital Object Identifier
doi:10.1214/aos/1176343886

Mathematical Reviews number (MathSciNet)
MR443204

Zentralblatt MATH identifier
0366.62051

JSTOR
links.jstor.org

Subjects
Primary: 62G05: Estimation
Secondary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]

Keywords
Regression function conditional quantities prediction multiple classification consistency in Bayes risk approximate Bayes rules nonparametric estimators nearest neighbor rules

Citation

Stone, Charles J. Consistent Nonparametric Regression. Ann. Statist. 5 (1977), no. 4, 595--620. doi:10.1214/aos/1176343886. https://projecteuclid.org/euclid.aos/1176343886


Export citation