Electronic Journal of Statistics

A two stage $k$-monotone B-spline regression estimator: Uniform Lipschitz property and optimal convergence rate

Teresa M. Lebair and Jinglai Shen

Full-text: Open access


This paper considers $k$-monotone estimation and the related asymptotic performance analysis over a suitable Hölder class for general $k$. A novel two stage $k$-monotone B-spline estimator is proposed: in the first stage, an unconstrained estimator with optimal asymptotic performance is considered; in the second stage, a $k$-monotone B-spline estimator is constructed (roughly) by projecting the unconstrained estimator onto a cone of $k$-monotone splines. To study the asymptotic performance of the second stage estimator under the sup-norm and other risks, a critical uniform Lipschitz property for the $k$-monotone B-spline estimator is established under the $\ell_{\infty }$-norm. This property uniformly bounds the Lipschitz constants associated with the mapping from a (weighted) first stage input vector to the B-spline coefficients of the second stage $k$-monotone estimator, independent of the sample size and the number of knots. This result is then exploited to analyze the second stage estimator performance and develop convergence rates under the sup-norm, pointwise, and $L_{p}$-norm (with $p\in [1,\infty )$) risks. By employing recent results in $k$-monotone estimation minimax lower bound theory, we show that these convergence rates are optimal.

Article information

Electron. J. Statist., Volume 12, Number 1 (2018), 1388-1428.

Received: May 2017
First available in Project Euclid: 17 May 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Asymptotic analysis B-splines convergence rates $k$-monotone estimation nonparametric regression shape constraints

Creative Commons Attribution 4.0 International License.


Lebair, Teresa M.; Shen, Jinglai. A two stage $k$-monotone B-spline regression estimator: Uniform Lipschitz property and optimal convergence rate. Electron. J. Statist. 12 (2018), no. 1, 1388--1428. doi:10.1214/18-EJS1426. https://projecteuclid.org/euclid.ejs/1526544023

Export citation


  • [1] Balabdaoui, F. and Wellner, J.A. (2010) Estimation of a $k$-monotone density: characterizations, consistency and minimax lower bounds., Stat. Neerl., 64, 45–70.
  • [2] Balabdaoui, F. and Wellner, J.A. (2007) Estimation of a $k$-monotone density: limit distribution theory and spline connection., Ann. Statist., 35, 2536–2564.
  • [3] Beatson, R.K. (1981) Convex approximation by splines., SIAM J. Math. Anal., 12, 549–559.
  • [4] Beatson, R.K. (1982) Monotone and convex approximation by splines: error estimates and a curve fitting algorithm., SIAM J. Numer. Anal., 19, 1278–1285.
  • [5] Bellec, P. and Tsybakov, A. (2015) Sharp oracle bounds for monotone and convex regression through aggregation., J. Mach. Learn. Res., 16, 1879–1892.
  • [6] Birke, M. and Dette, J. (2007) Estimating a convex function in nonparametric regression., Scand. J. Stat., 34, 384–404.
  • [7] Cator, E. (2011) Adaptivity and optimality of the monotone least squares estimator., Bernoulli, 17, 714–735.
  • [8] De Boor, C. (2001), A Practical Guide to Splines. Springer.
  • [9] DeVore, C. (1977) Monotone approximation by splines., SIAM J. Math. Anal., 8 891–905.
  • [10] DeVore, R.A. and Lorentz, G.G. (1993), Constructive Approximation. Springer.
  • [11] Dümbgen, L., Freitag, S., Jongbloed, G. (2004) Consistency of concave regression with an application to current-status data., Math. Methods Statist., 13, 69–81.
  • [12] Eeckhoudt, L. and Schlesinger, H. (2013) Higher-order risk attitudes. In, Handbook of Insurance 41–57. Springer.
  • [13] Gao, F.C. (2008) Entropy estimate for $k$-monotone functions via small ball probability of integrated Brownian motions., Electron. Commun. Probab., 13, 121–130.
  • [14] Gao, F.C. and Wellner, J.A. (2009) On the rate of convergence of the maximum likelihood estimator of a $k$-monotone density., Sci. China Ser. A, 52, 1–14.
  • [15] Giguelay, J. (2017) Estimation of a discrete probability under constraint of $k$-monotonicity., Electron. J. Stat., 11, 1–49.
  • [16] Golitschek, M.V. (2014) On the $L_\infty $-norm of the orthogonal projector onto splines: A short proof of A. Shardin’s theorem., J. Approx. Theory, 181, 30–42.
  • [17] Groeneboom, P., Jongbloed, F., and Wellner, J.A. (2001) Estimation of a convex function: Characterizations and asymptotic theory., Ann. Statist., 29, 1653–1698.
  • [18] Guntuboyina, A. and Sen, B. (2015) Global risk bounds and adaptation in univariate convex regression., Probab. Theory Related Fields, 63, 379–411.
  • [19] Hu, Y.K. (1991) Convexity preserving approximation by free knot splines., SIAM J. Math. Anal., 22 1183–1191.
  • [20] Konovalov, V.N. and Leviatan, D. (2001) Estimates on the approximation of 3-monotone functions by 3-monotone quadratic splines., East J. Approx., 7, 333–349.
  • [21] Konovalov, V.N. and Leviatan, D. (2003) Shape preserving widths of Sobolev-type classes of $s$-monotone functions on a finite interval., Israel J. Math., 133, 239–268.
  • [22] Kuipers, B.J., Chiu, C., Dalle Molle, D.T., and Throop, D.R. (1991). Higher-order derivative constraints in qualitative simulation., Artificial Intelligence, 51, 343–379.
  • [23] Lebair, T.M. (2016), Constrained Estimation and Approximation using Control, Optimization, and Spline Theory. Ph.D. Dissertation, University of Maryland, Baltimore County. http://pages.jh.edu/~tlebair1/Lebair_thesis.pdf
  • [24] Lebair, T.M., Shen, J., and Wang, X. (2016) Minimax lower bound and optimal estimation of convex functions in the sup-norm., IEEE Trans. Automat. Control, Vol. 62(7), pp. 3482–3487, 2017.
  • [25] Mammen, E. (1991) Nonparametric regression under qualitative smoothness assumptions., Ann. Statist., 19, 741–759.
  • [26] Mammen, E. and van de Geer, S. (1997) Locally adaptive regression splines., Ann. Statist., 25, 387–413.
  • [27] Nemirovski, A. (2000) Topics in Non-parametric Statistics. In, Ecole d’Eté de Probabilités de Saint-Flour, Berlin, Germany: Springer, 1738, Lecture Notes in Mathematics.
  • [28] Pal, J.K. (2008) Spiking problem in monotone regression: Penalized residual sum of squares., Statist. Probab. Lett., 78, 1548–1556.
  • [29] Pal, J.K. and Woodroofe, M. (2007) Large sample properties of shape restricted regression estimators with smoothness adjustments., Statist. Sinica, 17, 1601–1616.
  • [30] Prymak, A.V. (2002) Three-convex approximation by quadratic splines with arbitrary fixed knots., East J. Approx., 8, 185–196.
  • [31] Robertson, T., Wright, F.T., and Dykstra, R.L. (1988), Order Restricted Statistical Inference. John Wiley & Sons.
  • [32] Sanyal, A.K., Chellappa, M., Ahmed, J., Shen, J., and Bernstein, D.S. (2003) Globally convergent adaptive tracking of spacecraft angular velocity with inertia identification and adaptive linearization., Proc. of the 42nd IEEE Conf. on Dec. and Contr., 2704–2709, Hawaii, 2003.
  • [33] Scholtes, S. (2012), Introduction to piecewise differentiable equations. Springer.
  • [34] Shadrin, A.Y. (2001) The $L_\infty $-norm of the $L_2$-spline projector is bounded independently of the knot sequence: A proof of de Boor’s conjecture., Acta Math., 87, 59–137.
  • [35] Shen, J. and Lebair, T.M. (2015) Shape restricted smoothing splines via constrained optimal control and nonsmooth Newton’s methods., Automatica, 53, 216–224.
  • [36] Shen, J. and Wang, X. (2010) Estimation of shape constrained functions in dynamical systems and its application to genetic networks., Proc. of Am. Control Conf., 5948–5953, Baltimore.
  • [37] Shen, J. and Wang, X. (2011) Estimation of monotone functions via $P$-splines: A constrained dynamical optimization approach., SIAM J. Control Optim., 49, 646–671.
  • [38] Shen, J. and Wang, X. (2012) Convex regression via penalized splines: A complementarity approach., Proc. of Am. Control Conf., 332–337, Montreal, Canada.
  • [39] Tsybakov, A.B. (2010), Introduction to Nonparametric Estimation. Springer.
  • [40] Wang, X. and Shen, J. (2010) A class of grouped Brunk estimators and penalized spline estimators for monotone regression., Biometrika, 97, 585–601.
  • [41] Wang, X. and Shen, J. (2013) Uniform convergence and rate adaptive estimation of convex functions via constrained optimization., SIAM J. Control Optim., 51, 2753–2787.
  • [42] Woodroofe, M. and Sun, J. (1993) A penalized maximum likelihood estimate of $f(0_+)$ when $f$ is nonincreasing., Statist. Sinica, 3, 501–515.