The Annals of Statistics

Tensor product space ANOVA models

Yi Lin

Full-text: Open access


To deal with the curse of dimensionality in high-dimensional nonparametric problems, we consider using tensor product space ANOVA models, which extend the popular additive models and are able to capture interactions of any order. The multivariate function is given an ANOVA decomposition, that is, it is expressed as a constant plus the sum of functions of one variable (main effects), plus the sum of functions of two variables (two-factor interactions)and so on. We assume the interactions to be in tensor product spaces.We show in both regression and white noise settings, the optimal rate of convergence for the TPS-ANOVA model is within a log factor of the one-dimensional optimal rate, and that the penalized likelihood estimator in TPS-ANOVA achieves this rate of convergence. The quick optimal rate of the TPS-ANOVA model makes it very preferable in high-dimensional function estimation. Many properties of the tensor product space of Sobolev –Hilbert spaces are also given.

Article information

Ann. Statist. Volume 28, Number 3 (2000), 734-755.

First available in Project Euclid: 12 March 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62J20: Diagnostics

Functional ANOVA tensor product space white noise model rate of convergence optimal rate of convergence penalized likelihood estimation interaction curse of dimensionality smoothing splines


Lin, Yi. Tensor product space ANOVA models. Ann. Statist. 28 (2000), no. 3, 734--755. doi:10.1214/aos/1015951996.

Export citation


  • Aubin,J. P. (1979). Applied Functional Analysis. Wiley, New York.
  • Brown,L. D. and Low,M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann.Statist.24 2384-2398.
  • Chen,Z. (1991). Interaction spline models and their convergence rates. Ann.Statist.19 1855-1868.
  • Cox,D. D. (1988). Approximation of method of regularization estimators. Ann.Statist.16 694-712.
  • Cox,D. D. and O'Sullivan,F. (1990). Asymptotic analysis of penalized likelihood and related estimators. Ann.Statist.18 1676-1695.
  • Donoho,D. L.,Liu,R. C. and MacGibbon,B. (1990). Minimax risk over hyperrectangles, and implications. Ann.Statist.18 1416-1437.
  • Gu,C. and Qiu,C. (1993). Smoothing spline density estimation: theory. Ann.Statist.21 217-234.
  • Gu,C. and Qiu,C. (1994). Penalized likelihood regression: a simple asymptotic analysis. Statist. Sinica 4 297-304.
  • Gu,C. (1996). Penalized likelihood hazard estimation: a general procedure. Statist.Sinica 6 861-876.
  • Hastie,T. J. and Tibshirani,R. J. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Hastie,T. J. and Tibshirani,R. J. (1993). Varying-coefficient models. J.Roy.Statist.Soc.Ser.B 55 757-796.
  • Huber,P. J. (1981). Robust Statistics. Wiley, New York.
  • Kadison,R. V. and Ringrose,J. R. (1997). Fundamentals of the Theory of Operator Algebras. Amer. Math. Soc., Providence, RI.
  • Lin,Y. (1998). Tensor product space ANOVA models in multivariate function estimation. Ph.D. dissertation, Dept. Statistics, Univ. Pennsylvania.
  • Nussbaum,M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann.Statist.24 2399-2430.
  • Oden,J. T. and Reddy,J. N. (1976). An Introduction to the Mathematical Theory of Finite Elements. Wiley, New York.
  • O'Sullivan,F. (1993). Nonparametric estimation in the Cox model. Ann.Statist.21 124-145.
  • Silverman,B. W. (1982). On the estimation of a probability density function by the maximum penalised likelihood method. Ann.Statist.10 795-810.
  • Stone,C. J. (1980). Optimal rates of convergence for nonparametric estimators. Ann.Statist.8 1348-1360.
  • Stone,C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040-1053.
  • Stone,C. J. (1985). Additive regression and other nonparametric models. Ann.Statist.13 689-705.
  • Wahba,G. (1990). Spline models for Observational Data. SIAM, Philadelphia.
  • Wahba,G.,Wang,Y.,Gu,C.,Klein,R. and Klein,B. (1995). Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. Ann.Statist.23 1865-1895.
  • Weinberger,H. F. (1974). Variational Methods for Eigenvalue Approximation. SIAM, Philadelphia.