## The Annals of Statistics

### Local greedy approximation for nonlinear regression and neural network training

L. K. Jones

#### Abstract

A criterion for local estimation and approximation in nonlinear regres- sion and neural network training is introduced and motivated. $N$th-order greedy approximation for the regression (or target) function based on the criterion is shown to converge at rate $O(1/N^{1/2})$ in the nonsampling case.

#### Article information

Source
Ann. Statist., Volume 28, Number 5 (2000), 1379-1389.

Dates
First available in Project Euclid: 12 March 2002

https://projecteuclid.org/euclid.aos/1015957398

Digital Object Identifier
doi:10.1214/aos/1015957398

Mathematical Reviews number (MathSciNet)
MR1805788

Zentralblatt MATH identifier
1105.62354

Subjects
Primary: 62H99: None of the above, but in this section

#### Citation

Jones, L. K. Local greedy approximation for nonlinear regression and neural network training. Ann. Statist. 28 (2000), no. 5, 1379--1389. doi:10.1214/aos/1015957398. https://projecteuclid.org/euclid.aos/1015957398

#### References

• [1] Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 40 930-945.
• [2] Bottou, L. and Vapnik, V. (1992). Local learning algorithms. Neural Computation 4 888-900.
• [3] Blum, A. L. and Rivest, R. L. (1992). Training a 3-Node Neural Network is NP-Complete. Neural Networks 5 117-127.
• [4] DeVore, R. A. and Temlyakov, V. N. (1996). Some remarks on greedy algorithms. Adv. Comput. Math 5 173-187.
• [5] Donohue, M. J., Gurvitz, L., Darken, C. and Sontag, E. (1994). Rates of convex approximation in non-Hilbert spaces. Constr. Approx. 13 187-220.
• [6] Flick, T. E., Jones, L. K., Priest, R. and Herman, C.(1990). Projection pursuit classification. Pattern Recognition 23 1367-1376.
• [7] Friedman, J. H. (1999). Greedy function approximation: a gradient boosting machine. Technical report, Stanford Univ.
• [8] Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817-823.
• [9] Huber, P. J. (1985). Projection pursuit. Ann. Statist. 13 435-475.
• [10] Jones, L. K. (1987). On a conjuecture of Huber concerning the convergence of projection pursuit regression. Ann. Statist. 15 880-882.
• [11] Jones, L. K. (1992). A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Statist. 20 608-613.
• [12] Jones, L. K. (1994). Good weights and hyperbolic kernels for neural networks, projection pursuit, and pattern classification: Fourier strategies for extracting information from high-dimensional data. IEEE Trans. Inform. Theory 40 439-454.
• [13] Jones, L. K. (1997). The Computational intractability of training sigmoidal neural networks. IEEE Trans. Inform. Theory 43 167-173.
• [14] Lee, W. S. and Bartlett, P. L., and Williamson, R. C. (1996). Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 2118-2132.
• [15] Rejto, L. and Walter, G. G. (1992). Remarks on projection pursuit regression and density estimation. Stochastic Anal. Appl. 10 213-222.
• [16] Vu, V. H. (1998). On the infeasibility of training neural networks with small mean-squared error. IEEE Trans. Inform. Theory 44 2892-2900.