Electronic Journal of Statistics

Circumventing superefficiency: An effective strategy for distributed computing in non-standard problems

Moulinath Banerjee and Cécile Durot

Full-text: Open access

Abstract

We propose a strategy for computing estimators in some non-standard M-estimation problems, where the data are distributed across different servers and the observations across servers, though independent, can come from heterogeneous sub-populations, thereby violating the identically distributed assumption. Our strategy fixes the super-efficiency phenomenon observed in prior work on distributed computing in (i) the isotonic regression framework, where averaging several isotonic estimates (each computed at a local server) on a central server produces super-efficient estimates that do not replicate the properties of the global isotonic estimator, i.e. the isotonic estimate that would be constructed by transferring all the data to a single server, and (ii) certain types of M-estimation problems involving optimization of discontinuous criterion functions where M-estimates converge at the cube-root rate. The new estimators proposed in this paper work by smoothing the data on each local server, communicating the smoothed summaries to the central server, and then solving a non-linear optimization problem at the central server. They are shown to replicate the asymptotic properties of the corresponding global estimators, and also overcome the super-efficiency phenomenon exhibited by existing estimators.

Article information

Source
Electron. J. Statist., Volume 13, Number 1 (2019), 1926-1977.

Dates
Received: June 2018
First available in Project Euclid: 19 June 2019

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1560909646

Digital Object Identifier
doi:10.1214/19-EJS1559

Mathematical Reviews number (MathSciNet)
MR3964267

Zentralblatt MATH identifier
07080065

Subjects
Primary: 62G05: Estimation 62G20: Asymptotic properties 62G08: Nonparametric regression
Secondary: 62E20: Asymptotic distribution theory

Keywords
Cube-root asymptotics distributed computing isotonic regression local minimax risk superefficiency

Rights
Creative Commons Attribution 4.0 International License.

Citation

Banerjee, Moulinath; Durot, Cécile. Circumventing superefficiency: An effective strategy for distributed computing in non-standard problems. Electron. J. Statist. 13 (2019), no. 1, 1926--1977. doi:10.1214/19-EJS1559. https://projecteuclid.org/euclid.ejs/1560909646


Export citation

References

  • [1] Anevski, D., Hössjer, O., et al. (2006). A general asymptotic scheme for inference under order restrictions., The Annals of Statistics, 34(4):1874–1930.
  • [2] Azadbakhsh, M., Jankowski, H., and Gao, X. (2014). Computing confidence intervals for log-concave densities., Computational Statistics & Data Analysis, 75:248–264.
  • [3] Banerjee, M., Durot, C., Sen, B., et al. (2019). Divide and conquer in nonstandard problems and the super-efficiency phenomenon., The Annals of Statistics, 47(2):720–757.
  • [4] Banerjee, M. et al. (2007). Likelihood based inference for monotone response models., The Annals of Statistics, 35(3):931–956.
  • [5] Banerjee, M. and Wellner, J. A. (2001). Likelihood ratio tests for monotone functions., Annals of Statistics, pages 1699–1731.
  • [6] Battey, H., Fan, J., Liu, H., Lu, J., and Zhu, Z. (2018). Distributed testing and estimation under sparse high dimensional models., Annals of statistics, 46(3):1352.
  • [7] Billingsley, P. (2013)., Convergence of probability measures. John Wiley & Sons.
  • [8] Groeneboom, P., Jongbloed, G., Witte, B. I., et al. (2010). Maximum smoothed likelihood estimation and smoothed maximum likelihood estimation in the current status model., The Annals of Statistics, 38(1):352–387.
  • [9] Groeneboom, P. and Wellner, J. A. (2001). Computing Chernoff’s distribution., J. Comput. Graph. Statist., 10(2):388–400.
  • [10] Hsieh, C.-J., Si, S., and Dhillon, I. (2014). A divide-and-conquer solver for kernel support vector machines. In, International Conference on Machine Learning, pages 566–574.
  • [11] Kim, J. and Pollard, D. (1990). Cube root asymptotics., Annals of Statistics, 18:191–219.
  • [12] Li, R., Lin, D. K., and Li, B. (2013). Statistical inference in massive data sets., Applied Stochastic Models in Business and Industry, 29(5):399–409.
  • [13] Lu, J., Cheng, G., and Liu, H. (2016). Nonparametric heterogeneity testing for massive data., arXiv preprint arXiv:1601.06212.
  • [14] Mammen, E. (1991). Estimating a smooth monotone regression function., The Annals of Statistics, pages 724–740.
  • [15] Massart, P. (1990). The tight constant in the dvoretzky-kiefer-wolfowitz inequality., The Annals of Probability, pages 1269–1283.
  • [16] Mukerjee, H. (1988). Monotone nonparametric regression., The Annals of Statistics, pages 741–750.
  • [17] Robertson, T., Wright, F. T., and Dykstra, R. L. (1988)., Order restricted statistical inference. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons Ltd., Chichester.
  • [18] Rosenthal, H. P. (1970). On the subspaces of $L_p,(p>2)$ spanned by sequences of independent random variables., Israel Journal of Mathematics, 8(3):273–303.
  • [19] Shang, Z. and Cheng, G. (2017). Computational limits of a distributed algorithm for smoothing spline., The Journal of Machine Learning Research, 18(1):3809–3845.
  • [20] Shi, C., Lu, W., and Song, R. (2018). A massive data framework for m-estimators with cubic-rate., Journal of the American Statistical Association, 113(524):1698–1709.
  • [21] Tang, R., Banerjee, M., Kosorok, M. R., et al. (2012). Likelihood based inference for current status data on a grid: A boundary phenomenon and an adaptive inference procedure., The Annals of Statistics, 40(1):45–72.
  • [22] Van Der Vaart, A. and Van Der Laan, M. (2003). Smooth estimation of a monotone density., Statistics: A Journal of Theoretical and Applied Statistics, 37(3):189–203.
  • [23] van der Vaart, A. W. and Wellner, J. A. (1996)., Weak convergence and empirical processes. Springer Series in Statistics. Springer-Verlag, New York. With applications to statistics.
  • [24] Volgushev, S., Chao, S.-K., and Cheng, G. (2019). Distributed inference for quantile regression processes., The Annals of Statistics, 47(3):1634–1662.
  • [25] Zhang, R., Kim, J., and Woodroofe, M. (2001). Asymptotic analysis of isotonic estimation for grouped data., Journal of statistical planning and inference, 98(1–2):107–117.
  • [26] Zhang, Y., Duchi, J., and Wainwright, M. (2013). Divide and conquer kernel ridge regression. In, Conference on Learning Theory, pages 592–617.
  • [27] Zhao, T., Cheng, G., and Liu, H. (2016). A partially linear framework for massive heterogeneous data., Annals of statistics, 44(4):1400.