Electronic Journal of Statistics

Lower bound in regression for functional data by representation of small ball probabilities

André Mas

Full-text: Open access


The minimax rate for estimating the regression function $r(\cdot)=\mathbb{E}(y|X=\cdot)$ when $y\in\mathbb{R}$ and $X$ takes values in a function space is the initial motivation of this work. Recent articles underline the major role of the shifted small ball probability $\mathbb{P}(\Vert X-x_{0}\Vert <\cdot)$ in the variance of classical estimates. The main results are twofold. First, starting from a theorem by [41], we study the small ball probability $\mathbb{P}(S<\varepsilon)$ when $\varepsilon\downarrow0$ with $S=\sum_{i=1}^{+\infty}\lambda_{i}Z_{i}$ where the $Z_{i}$’s are i.i.d. positive and $(\lambda_{i})_{i\in\mathbb{N}}$ a positive nonincreasing sequence such that $\sum\lambda_{i}<+\infty$. It is shown that $\mathbb{P}(S<\cdot)$ belongs to a class of functions introduced by de Haan, well-known in extreme value theory, the class of Gamma-varying functions, for which an exponential-integral representation is available. Second this approach allows to derive minimax lower bounds for the risk at a fixed point $x_{0}$ when $X\in\mathcal{H}$ some Hilbert space of functions. Denoting this minimax risk: \[\mathcal{R}_{n}^{\ast}=\inf_{T_{n}}\sup_{r\in\mathcal{E}}\mathbb{E}\left\vert T_{n}-r(x_{0})\right\vert ^{2}\] where $T_{n}$ is any estimate of $r(x_{0})$ and $\mathcal{E}$ is some class of smooth functions from $\mathcal{H}$ to $\mathbb{R}$ it turns out that, in a general framework, $n^{\tau}\mathcal{R}_{n}^{\ast}\rightarrow+\infty$ for any $\tau>0.$ This negative result may pave the way towards new approaches for modeling regression with functional data.

Article information

Electron. J. Statist., Volume 6 (2012), 1745-1778.

First available in Project Euclid: 27 September 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]
Secondary: 62G20: Asymptotic properties

Functional data small ball problems nonparametric regression regular variation Gaussian random elements lower bound


Mas, André. Lower bound in regression for functional data by representation of small ball probabilities. Electron. J. Statist. 6 (2012), 1745--1778. doi:10.1214/12-EJS726. https://projecteuclid.org/euclid.ejs/1348753351

Export citation


  • [1] de Acosta A. (1983): Small deviations in the functional central limit theorem with applications to functional laws of the iterated logarithm, Ann. Probab., 11, 78–101.
  • [2] Ait-Saadi A., Ferraty F., Kassa R., Vieu P. (2008): Cross-validated estimations in the single-functional index model, Statistics, 42, 475–494.
  • [3] Aurzada F., Lifshits, M., Linde W. (2009): Small deviations of stable processes and entropy of associated random operators, Bernoulli, 15, 1305–1334.
  • [4] Berlinet A., Elamine A., Mas A. (2012): Local linear regression for functional data, Ann. Instit. Statist. Math., 63, 1047–1075.
  • [5] Bingham N.H., Goldie C.M. and Teugels J.L. (1987):, Regular Variations. Encyclopedia of Mathematics and Its Applications, Cambridge University Press.
  • [6] Bloom S. (1976): A characterization of B-slowly varying functions, Proc. Amer. Math. Soc., 54, 243–250.
  • [7] Borell C. (1976): Gaussian Radon measures on locally convex spaces, Math. Scand., 38, 265–285.
  • [8] Cai T., Hall P. (2006): Prediction in functional linear regression, Ann. Statist., 34, 2159–2179.
  • [9] Cardot H., Johannes J. (2010): Thresholding projection estimators in functional linear models, J. Multivariate Anal., 101, 395–408.
  • [10] Crambes C., Kneip A., Sarda P. (2009): Smoothing splines estimators for functional linear regression, Ann. Statist., 37, 35–72.
  • [11] Dabo-Niang S. (2002): Estimation de la densité dans un espace de dimension infinie: Application aux diffusions, C. R. Math. Acad. Sci. Paris., 334, 213–216.
  • [12] Delaigle A., Hall P. (2010): Defining probability for a distribution of random functions, Ann. Statist., 38, 1171–1193.
  • [13] Dembo A., Meyer-Wolf E., Zeitouni O. (1995): Exact behavior of Gaussian semi-norms, Statist. Probab. Lett., 23, 275–280.
  • [14] Dunker T., Lifshits M.A., Linde W. (1998): Small deviation probabilities of sums of independent random variables. In High Dimensional Probability (Oberwolfach, 1996), 59–74, Progr. Probab., 43, Birkhäuser, Basel.
  • [15] Erickson K. (1980): Rates of escape of infinite-dimensional Brownian motion, Ann. Probab., 8, 325–338.
  • [16] Fan, J. (1991): On the optimal rates of convergence for nonparametric deconvolution problems., Ann. Statist., 19, 1257–1272.
  • [17] Ferraty F., Hall P., Vieu P. (2010): Most-predictive design points for functional data predictors, Biometrika., 97 n$^\circ$34, 807–824.
  • [18] Ferraty F., Laksaci A., Tadj A., Vieu P. (2010): Rate of uniform consistency for nonparametric estimates with functional variables., J. Statist. Plann. Inference, 140, no. 2, 335–352.
  • [19] Ferraty F., Mas A., Vieu P. (2007): Advances on nonparametric regression from functional data, Aust. New Zealand J. Stat., 49 n$^\circ$3, 1–20.
  • [20] Ferraty F., van Keilegom I., Vieu P. (2010): On the validity of the boostrap in the nonparametric functional regression, Scand. J. Stat., 37, 286–306.
  • [21] Ferraty F., Vieu P. (2004): Nonparametric models for functional data, with applications in regression, time series prediction and curve discrimination., J. Nonparametr. Stat., 16, 111–125.
  • [22] Ferraty F., Vieu P. (2006):, Nonparametric Functional Data Analysis: Theory and Practice. Springer, Berlin.
  • [23] Ferraty F., Vieu P. (2009): Additive prediction and bosting for functional data, Comput. Statist. Data Anal., 53, 1400–1413.
  • [24] Ferraty F., Vieu P. (2010): Kernel regression estimation fur functional data. In, The Oxford Handbook of Functional data Analysis, Ferraty and Romain Eds, 72–129.
  • [25] Fisher R.A., Tippett L.H.C. (1928): Limiting forms of the frequency distribution of the largest or smaller member of a sample, Proceedings of the Cambridge Philosophical Society, 24, 180–190.
  • [26] Gaiffas S. (2005): Convergence rates for pointwise curve estimation with a degenerate design, Math. Methods Statist., 14, 1–27.
  • [27] Gardes L., Girard S., Lekina A. (2010): Functional nonparametric estimation of conditional extremal quantiles, J. Multivariate Anal., 101, 419–433.
  • [28] Gnedenko B.V. (1943): Sur la distribution limite du terme maximum d’une série aléatoire, Ann. of Math., 44, 423–453.
  • [29] de Haan L. (1971): A form of regular variation and its application to the domain of attraction of the double exponential distribution, Z. Wahrscheinlichkeitstheorie. verw. Geb., 17, 241–258.
  • [30] de Haan L. (1974): Equivalence classes of regularly varying functions, Stochastic Process. Appl., 2, 243–259.
  • [31] Hall P., Heckman N. (2002): Estimating and depicting the structure of a distribution of random functions, Biometrika, 89, 145–158.
  • [32] Hall P., Horowitz J.L.N. (2007): Methodology and convergence rate for functional linear regression., Ann. Statist., 35, 70–91.
  • [33] Jurek Z.J. (1979): A representation of the characteristic functions of stable distributions in Hilbert spaces., Banach Center Publications, 5 PWN Warsaw, 151–158.
  • [34] Kuelbs J., Li W.V., Linde W. (1994): Lim inf results for Gaussian samples and Chung’s functional LIL, Ann. Probab., 22, 1879–1903.
  • [35] Ledoux M., Talagrand M. (1991):, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete, 23, Springer-Verlag.
  • [36] Li W.V., Linde W. (1993): Small ball problems for non centered Gaussian measures, Probab. Math. Statist., 14, 231–251.
  • [37] Li W.V., Linde W. (1999): Approximation, metric entropy and small ball estimates for Gaussian measures., Ann. Probab., 27, 1556–1578.
  • [38] Li W.V., Linde W. (2004): Small deviations of stable processes via metric entropy, J. Theoret. Probab., 17, 261–284.
  • [39] Li, W.V., Shao Q.-M. (2001): Gaussian processes: Inequalities, small ball probabilities and applications, In Rao and Shanbhag (eds.), Handbook of Statistics, 19, Elsevier, New-York, 533–597.
  • [40] Lifshits M.A. (1995): Gaussian random functions. Mathematics and its Applications, 322. Kluwer Academic Publishers, Dordrecht.
  • [41] Lifshits M.A. (1997): On the lower tail probabilities of some random series, Ann. Probab., 25, 424–442.
  • [42] Mas A. (2008): Local functional principal component analysis, Complex Analysis and Operator Theory, 2, 135–167.
  • [43] Meyer-Wolf E., Zeitouni O. (1993): The probability of small Gaussian ellipsoïds, Ann. Probab., 21, 14–24.
  • [44] Müller H.G., Yan X. (2001): On local moments, J. Multivariate Anal., 76, 90–109.
  • [45] Müller H.G., Yao F. (2008): Functional additive models, J. Amer. Statist. Assoc., 103, 1534–1544.
  • [46] Rachdi M., Vieu P. (2007): Nonparametric regression for functional data: Automatic smoothing parameter selection, Journal of Statistical Planning and Inference, 137, 2784–2801.
  • [47] Ramsay, J.O. and Silverman, B.W. (2005):, Functional data analysis (Second Ed.). Springer.
  • [48] Skorohod, A.V. (1967): On the densities of probability measures in functional spaces. Proc. Fifth Berkeley Sympos. Math. Statist. and Probability (Berkeley, Calif., 1965/66) Vol. II: Contributions to Probability Theory, Part 1, p. 163–182 Univ. California Press, Berkeley, Calif.
  • [49] Stone C.J. (1980): Optimal rates of convergence for nonparametric estimators, Ann. Statist., 8, 1348–1360.
  • [50] Stone C.J. (1982): Optimal global rates of convergence for nonparametric regression., Ann. Statist., 10, 1040–1053.
  • [51] Stone C.J. (1985): Additive regression and other nonparametric models, Ann. Statist., 13, 689–705.
  • [52] Sytaya G.N. (1974): On some asymptotic representations of the Gaussian measure in a Hilbert space, Theory of Stochastic Processes, 2, 93–104.
  • [53] Talagrand M. (1992): On the rate of convergence in Strassen’s LIL, Progr. Probab., Birkhauser, Boston, 339–351.
  • [54] Tsybakov A. (2004): Introduction à l’estimation non-paramétrique. Mathématiques et applications., Springer.
  • [55] van der Vaart, A.W., van Zanten, J.H. (2007): Bayesian inference with rescaled Gaussian process priors, Electronic J. Statist., 1, 433–448.
  • [56] van der Vaart, A.W., van Zanten, J.H. (2008): Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist., 36, 1435–1463.