Bernoulli

  • Bernoulli
  • Volume 16, Number 2 (2010), 543-560.

Asymptotic minimax risk of predictive density estimation for non-parametric regression

Xinyi Xu and Feng Liang

Full-text: Open access

Abstract

We consider the problem of estimating the predictive density of future observations from a non-parametric regression model. The density estimators are evaluated under Kullback–Leibler divergence and our focus is on establishing the exact asymptotics of minimax risk in the case of Gaussian errors. We derive the convergence rate and constant for minimax risk among Bayesian predictive densities under Gaussian priors and we show that this minimax risk is asymptotically equivalent to that among all density estimators.

Article information

Source
Bernoulli, Volume 16, Number 2 (2010), 543-560.

Dates
First available in Project Euclid: 25 May 2010

Permanent link to this document
https://projecteuclid.org/euclid.bj/1274821083

Digital Object Identifier
doi:10.3150/09-BEJ222

Mathematical Reviews number (MathSciNet)
MR2668914

Zentralblatt MATH identifier
1345.62065

Keywords
asymptotic minimax risk convergence rate non-parametric regression Pinsker’s theorem predictive density

Citation

Xu, Xinyi; Liang, Feng. Asymptotic minimax risk of predictive density estimation for non-parametric regression. Bernoulli 16 (2010), no. 2, 543--560. doi:10.3150/09-BEJ222. https://projecteuclid.org/euclid.bj/1274821083


Export citation

References

  • [1] Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547–554.
  • [2] Aslan, M. (2006). Asymptotically minimax Bayes predictive densities. Ann. Statist. 34 2921–2938.
  • [3] Belitser, E.N. and Levit, B.Y. (1995). On minimax filtering over ellipsoids. Math. Methods Statist. 3 259–273.
  • [4] Belitser, E.N. and Levit, B.Y. (1996). Asymptotically minimax nonparametric regression in L2. Statistics 28 105–122.
  • [5] Brown, L.D., George, E.I. and Xu, X. (2008). Admissible predictive density estimation. Ann. Statist. 36 1156–1170.
  • [6] Brown, L.D. and Low, M.G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384–2398.
  • [7] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist. 7 269–281.
  • [8] Efromovich, S.Y. (1994). On adaptive estimation of nonlinear functionals. Statist. Probab. Lett. 19 57–63.
  • [9] Efromovich, S.Y. (1999). Nonparametric Curve Estimation: Methods, Theory and Applications. New York: Springer.
  • [10] Efromovich, S.Y. and Pinsker, M.S. (1982). Estimation of square-integrable probability density of a random variable. Probl. Inf. Transm. 18 175–189.
  • [11] George, E.I., Liang, F. and Xu, X. (2006). Improved minimax prediction under Kullback–Leibler loss. Ann. Statist. 34 78–91.
  • [12] George, E.I. and Xu, X. (2008). Predictive density estimation for multiple regression. Econometric Theory 24 1–17.
  • [13] Goldenshluger, A. and Tsybakov, A.B. (2003). Optimal prediction for linear regression with infinitely many parameters. J. Multivariate Anal. 84 40–60.
  • [14] Golubev, G.K. and Nussbaum, M. (1990). A risk bound in Sobolev class regression. Ann. Statist. 18 758–778.
  • [15] Ibragimov, I.A. and Has’minskii, R.Z. (1977). On the estimation of an infinite-dimensional parameter in Gaussian white noise. Soviet Math. Dokl. 236 1053–1055.
  • [16] Ibragimov, I.A. and Has’minskii, R.Z. (1984). On nonparametric estimation of values of a linear functional in a Gaussian white noise. Teor. Veroyatn. Primen. 29 19–32.
  • [17] Johnstone, I.M. (2003). Function estimation and Gaussian sequence models. Draft of a Monograph.
  • [18] Liang, F. and Barron, A. (2004). Exact minimax strategies for predictive density estimation, data compression and model selection. IEEE Trans. Inform. Theory 50 2708–2726.
  • [19] Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.
  • [20] Nussbaum, M. (1999). Minimax risk: Pinsker bound. In Encyclopedia of Statistical Sciences (S. Kotz, ed.) 451–460. New York: Wiley.
  • [21] Pinsker, M.S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Probl. Inf. Transm. 2 120–133.
  • [22] Tsybakov, A.B. (1997). On nonparametric estimation of density level sets. Ann. Statist. 25 948–969.