The Annals of Statistics

Nonparametric regression in exponential families

Lawrence D. Brown, T. Tony Cai, and Harrison H. Zhou

Full-text: Open access


Most results in nonparametric regression theory are developed only for the case of additive noise. In such a setting many smoothing techniques including wavelet thresholding methods have been developed and shown to be highly adaptive. In this paper we consider nonparametric regression in exponential families with the main focus on the natural exponential families with a quadratic variance function, which include, for example, Poisson regression, binomial regression and gamma regression. We propose a unified approach of using a mean-matching variance stabilizing transformation to turn the relatively complicated problem of nonparametric regression in exponential families into a standard homoscedastic Gaussian regression problem. Then in principle any good nonparametric Gaussian regression procedure can be applied to the transformed data. To illustrate our general methodology, in this paper we use wavelet block thresholding to construct the final estimators of the regression function. The procedures are easily implementable. Both theoretical and numerical properties of the estimators are investigated. The estimators are shown to enjoy a high degree of adaptivity and spatial adaptivity with near-optimal asymptotic performance over a wide range of Besov spaces. The estimators also perform well numerically.

Article information

Ann. Statist., Volume 38, Number 4 (2010), 2005-2046.

First available in Project Euclid: 11 July 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Adaptivity asymptotic equivalence exponential family James–Stein estimator nonparametric Gaussian regression quadratic variance function quantile coupling wavelets


Brown, Lawrence D.; Cai, T. Tony; Zhou, Harrison H. Nonparametric regression in exponential families. Ann. Statist. 38 (2010), no. 4, 2005--2046. doi:10.1214/09-AOS762.

Export citation


  • Anscombe, F. J. (1948). The transformation of Poisson, binomial and negative binomial data. Biometrika 35 246–254.
  • Antoniadis, A. and Leblanc, F. (2000). Nonparametric wavelet regression for binary response. Statistics 34 183–213.
  • Antoniadis, A. and Sapatinas, T. (2001). Wavelet shrinkage for natural exponential families with quadratic variance functions. Biometrika 88 805–820.
  • Bar-Lev, S. K. and Enis, P. (1990). On the construction of classes of variance stabilizing transformations. Statist. Probab. Lett. 10 95–100.
  • Bartlett, M. S. (1936). The square root transformation in analysis of variance. J. Roy. Statist. Soc. Suppl. 3 68–78.
  • Berk, R. A. and MacDonald, J. (2008). Overdispersion and Poisson regression. J. Quantitative Criminology 4 289–308.
  • Besbeas, P., De Feis, I. and Sapatinas, T. (2004). A comparative simulation study of wavelet shrinkage estimators for poisson counts. Internat. Statist. Rev. 72 209–237.
  • Brown, L. D. (1986). Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory. IMS, Hayward, CA.
  • Brown, L. D., Cai, T. T., Zhang, R., Zhao, L. H. and Zhou, H. H. (2010). The Root-unroot algorithm for density estimation as implemented via wavelet block thresholding. Probab. Theory Related Fields 146 401–433.
  • Brown, L. D., Cai, T. T. and Zhou, H. H. (2008). Robust Nonparametric Estimation via Wavelet Median Regression. Ann. Statist. 36 2055–2084.
  • Brown L. D. and Low, M. G. (1996). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524–2535.
  • Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924.
  • Cai, T. T. (2002). On block thresholding in wavelet regression: Adaptivity, block Size, and threshold level. Statist. Sinica 12 1241–1273.
  • Cai, T. T. and Silverman, B. W. (2001). Incorporating information on neighboring coefficients into wavelet estimation. Sankhyā Ser. B 63 127–148.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia, PA.
  • DeVore, R. and Popov, V. (1988). Interpolation of Besov spaces. Trans. Amer. Math. Soc. 305 397–414.
  • Donoho, D. L. (1993). Nonlinear wavelet methods for recovery of signals, densities, and spectra from indirect and noisy data. In Different Perspectives on Wavelets (I. Daubechies, ed.) 173–205. Amer. Math. Soc., Providence, RI.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
  • Efron, B. (1982). Transformation theory: How normal is a family of a distributions? Ann. Statist. 10 323–339.
  • Fryźlewicz, P. and Nason, G. P. (2001). Poisson intensity estimation using wavelets and the Fisz transformation. J. Comput. Graph. Statist. 13 621–638.
  • Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922–942.
  • Hilbe, J. M. (2007). Negative Binomial Regression. Cambridge Univ. Press, Cambridge.
  • Hoyle, M. H. (1973). Transformations—an introduction and bibliography. Internat. Statist. Rev. 41 203–223.
  • Johnson, N. L., Kemp, A. W. and Kotz, S. (2005). Univariate Discrete Distributions. Wiley, New York.
  • Kaneko, Y. (2005). Spectral studies of Gamma-Ray burst prompt emission. Ph. D. dissertation, Univ. Alabama, Huntsville.
  • Kolaczyk, E. D. (1999a). Wavelet shrinkage estimation of certain Poisson intensity signals using corrected thresholds. Statist. Sinica 9 119–135.
  • Kolaczyk, E. D. (1999b). Bayesian multiscale models for Poisson processes. J. Amer. Statist. Assoc. 94 920–933.
  • Kolaczyk, E. D. and Nowak, R. D. (2005). Multiscale generalized linear models for nonparametric function estimation. Biometrika 92 119–133.
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent rv’s, and the sample df. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Lepski, O. V. (1990). On a problem of adaptive estimation in white Gaussian noise. Theory Probab. Appl. 35 454–466.
  • Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses, 3rd ed. Springer, Berlin.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Univ. Press, Cambridge.
  • Morris, C. (1982). Natural exponential families with quadratic variance functions. Ann. Statist. 10 65–80.
  • Petrov, V. V. (1975). Sums of Independent Random Variables. Springer, Berlin.
  • Pollard, D. P. (2002). A User’s Guide to Measure Theoretic Probability. Cambridge Univ. Press, Cambridge.
  • Quilligan, F., McBreen, B., Hanlon, L., McBreen, S., Hurley, K. J. and Watson, D. (2002). Temporal properties of gamma-ray bursts as signatures of jets from the central engine. Astronom. Astrophys. 385 377–398.
  • Runst, T. (1986). Mapping properties of non-linear operators in spaces of Triebel–Lizorkin and Besov type. Anal. Math. 12 313–346.
  • Strang, G. (1992). Wavelet and dilation equations: A brief introduction. SIAM Rev. 31 614–627.
  • Triebel, H. (1992). Theory of Function Spaces. II. Birkhäuser, Basel.
  • Yajnik, M., Moon, S. Kurose, J. and Towsley, D. (1999). Measurement and modelling of the temporal dependence in packet loss. In IEEE INFOCOM ’99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320) 345–353.
  • Zhou, H. H. (2006). A note on quantile coupling inequalities and their applications. Submitted. Available at