Electronic Journal of Statistics

Local optimization-based statistical inference

Shifeng Xiong

Full-text: Open access


This paper introduces a local optimization-based approach to test statistical hypotheses and to construct confidence intervals. This approach can be viewed as an extension of bootstrap, and yields asymptotically valid tests and confidence intervals as long as there exist consistent estimators of unknown parameters. We present simple algorithms including a neighborhood bootstrap method to implement the approach. Several examples in which theoretical analysis is not easy are presented to show the effectiveness of the proposed approach.

Article information

Electron. J. Statist., Volume 11, Number 1 (2017), 2295-2320.

Received: November 2016
First available in Project Euclid: 27 May 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F03: Hypothesis testing 62F25: Tolerance and confidence regions 62F40: Bootstrap, jackknife and other resampling methods

Bootstrap importance sampling non-regular problem resampling space-filling design stochastic programming

Creative Commons Attribution 4.0 International License.


Xiong, Shifeng. Local optimization-based statistical inference. Electron. J. Statist. 11 (2017), no. 1, 2295--2320. doi:10.1214/17-EJS1292. https://projecteuclid.org/euclid.ejs/1495850626

Export citation


  • [1] Andrews, D. W. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space., Econometrica 68 399–405.
  • [2] Bickel, P. J., Götze F, and van Zwet, W. R. (1997). Resampling fewer than n observations gains, losses, and remedies for losses., Statistica Sinica 7 1–31.
  • [3] Bickel, P. J. and Ren, J-J. (2001). The bootstrap in hypothesis testing, Lecture Notes-Monograph Series., State of the Art in Probability and Statistics 36 91–112.
  • [4] Blischke, W. R. (1974). On nonregular estimation. II. Estimation of the location parameter of the gamma and Weibull distributions., Communications in Statistics 3 1109–1129.
  • [5] Boyd, S. and Vandenberghe, L. (2004)., Convex Optimization. Cambridge University Press, Cambridge.
  • [6] Cheng, R. C. H. and Amin, N. A. K. (1983). Estimating parameters in continuous univariate distributions with a shifted origin., Journal of the Royal Statistical Society, Ser. B 45 394–403.
  • [7] Cousineau, D. (2009). Fitting the three-parameter Weibull distribution: Review and evaluation of existing and new methods., IEEE Transactions on Dielectrics and Electrical Insulation 16 281–288.
  • [8] Cox, D. R. and Oakes, D. (1984)., Analysis of Survival Data. Chapman & Hall, New York..
  • [9] Davison, A. C. and Hinkley, D. V. (1997)., Bootstrap Methods and Their Application. Cambridge University Press, Cambridge.
  • [10] de Carvalho, M. (2011). Confidence intervals for the minimum of a function using extreme value statistics., International Journal of Mathematical Modelling & Numerical Optimisation 2 288–296.
  • [11] de Haan, L. (1981). Estimation of the minimum of a function using order statistics., Journal of the American Statistical Association 76 467–469.
  • [12] Efron, B. (1979). Bootstrap methods: Another look at the jackknife., Annals of Statistics 7 1–26.
  • [13] Efron, B., Hastie, T., Johnstone, L., and Tibshirani, R. (2004). Least angle regression (with discussion)., Annals of Statistics 32 407–451.
  • [14] Ethier, S. N. (1982). Testing for favorable numbers on a roulette wheel., Journal of the American Statistical Association 77 660–665.
  • [15] Fan, J., Guo, S., and Hao, N. (2012). Variance estimation using refitted cross-validation in ultrahigh dimensional regression., Journal of the Royal Statistical Society, Ser. B 74 37–65.
  • [16] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties., Journal of the American Statistical Association 96 1348–1360.
  • [17] Fan, J., Zhang, C., and Zhang J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon., Annals of statistics 29 153–193.
  • [18] Fang, K. T., Hickernell, F. J., and Winker, P. (1996). Some global optimization algorithms in statistics, in, Lecture Notes in Operations Research, eds. by Du, D. Z., Zhang, X. S. and Cheng, K. World Publishing Corporation, 14–24.
  • [19] Fang, K. T., Lin, D. K. J., Winker, P., and Zhang, Y. (2000). Uniform design: theory and application., Technometrics 42 237–248.
  • [20] Fisher, R. A. (1959)., Statistical Methods and Scientific Inference, 2nd ed., revised. Hafner Publishing Company, New York.
  • [21] Frank, L. E. and Heiser, W. J. (2008). Feature selection in feature network models: Finding predictive subsets of features with the positive Lasso., British Journal of Mathematical and Statistical Psychology 61 1–27.
  • [22] Gelfand, A. E., Glaz, J., Kuo, L., and Lee, T. M. (1992). Inference for the maximum cell probability under multinomial sampling., Naval Research Logistics 39 97–114.
  • [23] Ghosh, J. K., Delampady, M., and Samanta, T. (2007)., An Introduction to Bayesian Analysis: Theory and Methods. Springer, New York.
  • [24] Glaz, J. and Sison, C. P. (1999). Simultaneous confidence intervals for multinomial proportions., Journal of Statistical Planning and Inference 82 251–262.
  • [25] Hall, P. (1992)., The bootstrap and Edgeworth Expansion. Springer, New York.
  • [26] Hannig, J., Iyer, H., and Patterson, P. (2006). Fiducial feneralized confidence intervals., Journal of the American Statistical Association 101 254–269.
  • [27] Härdle, W. and Luckhaus, S. (1984). Uniform consistency of a class of regression function estimators., Annals of Statistics 12 612–623.
  • [28] Hart, J. D. (1997)., Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York.
  • [29] Homem-de-Mello T. (2008). On rates of convergence for stochastic optimization problems under non-independent and identically distributed sampling., SIAM Journal on Optimization 19 524–551.
  • [30] Johnson, M. E., Moore, L. M., and Ylvisaker, D. (1990). Minimax and maximin distance designs., Journal of Statistical Planning and Inference 26 131–148.
  • [31] Kushner, H. J. and Yin, G. G. (1997)., Stochastic Approximation Algorithms and Applications. Springer, New York.
  • [32] Lehmann, E. L. and Romano, J. P. (2006)., Testing Statistical Hypotheses, 3rd., Springer, Science & Business Media.
  • [33] Lockhart, R. A. and Stephens, M. A. (1994). Estimation and tests of fit for the three-parameter Weibull distribution., Journal of the Royal Statistical Society, Ser. B 56 491–500.
  • [34] Martin, R. (2012). Plausibility functions and exact frequentist inference., Journal of the American Statistical Association 110 1552–1561.
  • [35] McKay, M. D., Beckman, R. J., and Conover, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21 239–245.
  • [36] Meng, X-L. (1994). Posterior predictive $p$-values., Annals of Statistics 22 1142–1160.
  • [37] Murthy, D. N. P., Xie, M., and Jiang, R. (2004)., Weibull Models. John Wiley & Sons, Hoboken, New Jersey.
  • [38] Park, J. S. (2001). Optimal Latin-hypercube designs for computer experiments., Journal of Statistical Planning Inference 39 15–111.
  • [39] Patil, G. P. and Taillie, C. (1979). An overview of diversity., Ecological Diversity in Theory and Practice, ed. by Grassle, J. F. et al. International Co-operative Publishing House. Fairland, MD.
  • [40] Politis, D. N., Romano, J. P., and Wolf, M. (1999)., Subsampling. Springer, New York.
  • [41] Shao, J. and Tu, D. (1995)., The Jackknife and Bootstrap. Springer, New York.
  • [42] Shapiro, A. (2003). Monte Carlo sampling methods, in Stochastic Programming., Handbook in OR & MS, Vol. 10, ed. by Ruszczyński, A. and Shapiro, A., North-Holland, Amsterdam.
  • [43] Silvapulle, M. J. and Sen, P. K. (2011)., Constrained Statistical Inference: Order, Inequality, and Shape Constraints. John Wiley & Sons, Hoboken, New Jersey.
  • [44] Teimouri, M., Hoseini, S. M., and Nadarajah, S. (2013). Comparison of estimation methods for the Weibull distribution., Statistics: A Journal of Theoretical and Applied Statistics 47 93–109.
  • [45] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso., Journal of the Royal Statistical Society, Ser. B 58 267–288.
  • [46] Weerahandi, S. (1995)., Exact Statistical Methods for Data Analysis. Springer, New York.
  • [47] Wu, L., Yang, Y., and Liu, H. (2014). Nonnegative-lasso and application in index tracking., Computational Statistics & Data Analysis 70 116–126
  • [48] Xie, M., Singh, K., and Strawderman, W. E. (2011). Confidence distributions and a unifying framework for meta-analysis., Journal of the American Statistical Association 106 320–333.
  • [49] Xiong, S. and Li, G. (2009). Inference for ordered parameters in multinomial distributions., Science in China, Ser. A: Mathematics 52 526–538.
  • [50] Xiong, S. and Li, G. (2010). Dispersion comparisons of two probability vectors under multinomial sampling., Acta Mathematica Scientia 30 907–918.
  • [51] Zhang. C-H. (2010). Nearly unbiased variable selection under minimax concave penalty., Annals of Statistics 38 894–942.