The Annals of Statistics

Local greedy approximation for nonlinear regression and neural network training

L. K. Jones

Full-text: Open access

Abstract

A criterion for local estimation and approximation in nonlinear regres- sion and neural network training is introduced and motivated. $N$th-order greedy approximation for the regression (or target) function based on the criterion is shown to converge at rate $O(1/N^{1/2})$ in the nonsampling case.

Article information

Source
Ann. Statist., Volume 28, Number 5 (2000), 1379-1389.

Dates
First available in Project Euclid: 12 March 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1015957398

Digital Object Identifier
doi:10.1214/aos/1015957398

Mathematical Reviews number (MathSciNet)
MR1805788

Zentralblatt MATH identifier
1105.62354

Subjects
Primary: 62H99: None of the above, but in this section

Keywords
Greedy approximation local training

Citation

Jones, L. K. Local greedy approximation for nonlinear regression and neural network training. Ann. Statist. 28 (2000), no. 5, 1379--1389. doi:10.1214/aos/1015957398. https://projecteuclid.org/euclid.aos/1015957398


Export citation

References

  • [1] Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 40 930-945.
  • [2] Bottou, L. and Vapnik, V. (1992). Local learning algorithms. Neural Computation 4 888-900.
  • [3] Blum, A. L. and Rivest, R. L. (1992). Training a 3-Node Neural Network is NP-Complete. Neural Networks 5 117-127.
  • [4] DeVore, R. A. and Temlyakov, V. N. (1996). Some remarks on greedy algorithms. Adv. Comput. Math 5 173-187.
  • [5] Donohue, M. J., Gurvitz, L., Darken, C. and Sontag, E. (1994). Rates of convex approximation in non-Hilbert spaces. Constr. Approx. 13 187-220.
  • [6] Flick, T. E., Jones, L. K., Priest, R. and Herman, C.(1990). Projection pursuit classification. Pattern Recognition 23 1367-1376.
  • [7] Friedman, J. H. (1999). Greedy function approximation: a gradient boosting machine. Technical report, Stanford Univ.
  • [8] Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817-823.
  • [9] Huber, P. J. (1985). Projection pursuit. Ann. Statist. 13 435-475.
  • [10] Jones, L. K. (1987). On a conjuecture of Huber concerning the convergence of projection pursuit regression. Ann. Statist. 15 880-882.
  • [11] Jones, L. K. (1992). A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Statist. 20 608-613.
  • [12] Jones, L. K. (1994). Good weights and hyperbolic kernels for neural networks, projection pursuit, and pattern classification: Fourier strategies for extracting information from high-dimensional data. IEEE Trans. Inform. Theory 40 439-454.
  • [13] Jones, L. K. (1997). The Computational intractability of training sigmoidal neural networks. IEEE Trans. Inform. Theory 43 167-173.
  • [14] Lee, W. S. and Bartlett, P. L., and Williamson, R. C. (1996). Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 2118-2132.
  • [15] Rejto, L. and Walter, G. G. (1992). Remarks on projection pursuit regression and density estimation. Stochastic Anal. Appl. 10 213-222.
  • [16] Vu, V. H. (1998). On the infeasibility of training neural networks with small mean-squared error. IEEE Trans. Inform. Theory 44 2892-2900.