Bernoulli

  • Bernoulli
  • Volume 24, Number 2 (2018), 1072-1100.

On matrix estimation under monotonicity constraints

Sabyasachi Chatterjee, Adityanand Guntuboyina, and Bodhisattva Sen

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We consider the problem of estimating an unknown $n_{1}\times n_{2}$ matrix $\mathbf{\theta}^{*}$ from noisy observations under the constraint that $\mathbf{\theta}^{*}$ is nondecreasing in both rows and columns. We consider the least squares estimator (LSE) in this setting and study its risk properties. We show that the worst case risk of the LSE is $n^{-1/2}$, up to multiplicative logarithmic factors, where $n=n_{1}n_{2}$ and that the LSE is minimax rate optimal (up to logarithmic factors). We further prove that for some special $\mathbf{\theta}^{*}$, the risk of the LSE could be much smaller than $n^{-1/2}$; in fact, it could even be parametric, that is, $n^{-1}$ up to logarithmic factors. Such parametric rates occur when the number of “rectangular” blocks of $\mathbf{\theta}^{*}$ is bounded from above by a constant. We also derive an interesting adaptation property of the LSE which we term variable adaptation – the LSE adapts to the “intrinsic dimension” of the problem and performs as well as the oracle estimator when estimating a matrix that is constant along each row/column. Our proofs, which borrow ideas from empirical process theory, approximation theory and convex geometry, are of independent interest.

Article information

Source
Bernoulli, Volume 24, Number 2 (2018), 1072-1100.

Dates
Received: November 2015
Revised: April 2016
First available in Project Euclid: 21 September 2017

Permanent link to this document
https://projecteuclid.org/euclid.bj/1505980890

Digital Object Identifier
doi:10.3150/16-BEJ865

Mathematical Reviews number (MathSciNet)
MR3706788

Zentralblatt MATH identifier
06778359

Keywords
adaptation bivariate isotonic regression metric entropy bounds minimax lower bound oracle inequalities tangent cone variable adaptation

Citation

Chatterjee, Sabyasachi; Guntuboyina, Adityanand; Sen, Bodhisattva. On matrix estimation under monotonicity constraints. Bernoulli 24 (2018), no. 2, 1072--1100. doi:10.3150/16-BEJ865. https://projecteuclid.org/euclid.bj/1505980890


Export citation

References

  • [1] Ayer, M., Brunk, H.D., Ewing, G.M., Reid, W.T. and Silverman, E. (1955). An empirical distribution function for sampling with incomplete information. Ann. Math. Stat. 26 641–647.
  • [2] Barlow, R.E., Bartholomew, D.J., Bremner, J.M. and Brunk, H.D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression. New York: Wiley.
  • [3] Bellec, P. (2015). Sharp oracle inequalities for least squares estimators in shape restricted regression. Preprint. Available at arXiv:1510.08029.
  • [4] Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
  • [5] Brunk, H.D. (1955). Maximum likelihood estimates of monotone parameters. Ann. Math. Stat. 26 607–616.
  • [6] Chatterjee, S. (2014). A new perspective on least squares under convex constraint. Ann. Statist. 42 2340–2381.
  • [7] Chatterjee, S. (2015). Matrix estimation by universal singular value thresholding. Ann. Statist. 43 177–214.
  • [8] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist. 43 1774–1800.
  • [9] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On matrix estimation under monotonicity constraints. Available at http://arxiv.org/abs/1506.03430.
  • [10] Chatterjee, S. and Mukherjee, S. (2016). On estimation in tournaments and graphs under monotonicity constraints. Preprint. Available at arXiv:1603.04556.
  • [11] Donoho, D. (1991). Gelfand $n$-widths and the method of least squares Technical report, University of California, Berkeley.
  • [12] Gao, F. and Wellner, J.A. (2007). Entropy estimate for high-dimensional monotonic functions. J. Multivariate Anal. 98 1751–1764.
  • [13] Gebhardt, F. (1970). An algorithm for monotone regression with one or more independent variables. Biometrika 57 263–271.
  • [14] Hanson, D.L., Pledger, G. and Wright, F.T. (1973). On consistency in monotonic regression. Ann. Statist. 1 401–421.
  • [15] Makowski, G.G. (1977). Consistency of an estimator of doubly nondecreasing regression functions. Z. Wahrsch. Verw. Gebiete 39 263–268.
  • [16] Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083–1104.
  • [17] Oymak, S. and Hassibi, B. (2016). Sharp MSE Bounds for Proximal Denoising. Found. Comput. Math. 16 965–1029.
  • [18] Robertson, T. and Wright, F.T. (1975). Consistency in generalized isotonic regression. Ann. Statist. 3 350–362.
  • [19] Robertson, T., Wright, F.T. and Dykstra, R.L. (1988). Order Restricted Statistical Inference. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. Chichester: Wiley.
  • [20] Shah, N.B., Balakrishnan, S., Guntuboyina, A. and Wainright, M.J. (2015). Stochastically transitive models for pairwise comparisons: Statistical and computational issues. Preprint. Available at arXiv:1510.05610.
  • [21] van Eeden, C. (1958). Testing and Estimating Ordered Parameters of Probability Distributions Amsterdam: Mathematical Centre.
  • [22] van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907–924.
  • [23] van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14–44.
  • [24] van de Geer, S. and Wainwright, M. (2015). On concentration for (regularized) empirical risk minimization. Preprint. Available at arXiv:1512.00677.
  • [25] van de Geer, S.A. (2000). Applications of Empirical Process Theory. Cambridge Series in Statistical and Probabilistic Mathematics 6. Cambridge: Cambridge Univ. Press.
  • [26] van der Vaart, A.W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge: Cambridge Univ. Press.
  • [27] van der Vaart, A.W. and Wellner, J.A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. New York: Springer.
  • [28] Wang, Y. (1996). The $L_{2}$ risk of an isotonic estimate. Comm. Statist. Theory Methods 25 281–294.
  • [29] Zhang, C.-H. (2002). Risk bounds in isotonic regression. Ann. Statist. 30 528–555.