The Annals of Statistics

Higher order semiparametric frequentist inference with the profile sampler

Guang Cheng and Michael R. Kosorok

Full-text: Open access


We consider higher order frequentist inference for the parametric component of a semiparametric model based on sampling from the posterior profile distribution. The first order validity of this procedure established by Lee, Kosorok and Fine in [J. American Statist. Assoc. 100 (2005) 960–969] is extended to second-order validity in the setting where the infinite-dimensional nuisance parameter achieves the parametric rate. Specifically, we obtain higher order estimates of the maximum profile likelihood estimator and of the efficient Fisher information. Moreover, we prove that an exact frequentist confidence interval for the parametric component at level α can be estimated by the α-level credible set from the profile sampler with an error of order OP(n−1). Simulation studies are used to assess second-order asymptotic validity of the profile sampler. As far as we are aware, these are the first higher order accuracy results for semiparametric frequentist inference.

Article information

Ann. Statist., Volume 36, Number 4 (2008), 1786-1818.

First available in Project Euclid: 16 July 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G20: Asymptotic properties 62F25: Tolerance and confidence regions
Secondary: 62F15: Bayesian inference 62F12: Asymptotic properties of estimators

Higher order frequentist inference posterior distribution Markov chain Monte Carlo profile likelihood Cox proportional hazards model proportional odds model case-control studies with a missing covariate


Cheng, Guang; Kosorok, Michael R. Higher order semiparametric frequentist inference with the profile sampler. Ann. Statist. 36 (2008), no. 4, 1786--1818. doi:10.1214/07-AOS523.

Export citation


  • [1] Begun, J. M., Hall, W. J., Huang, W.-.M. and Wellner, J. A. (1983). Information and asymptotic efficiency in parametric–nonparametric models. Ann. Statist. 11 432–452.
  • [2] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1998). Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York.
  • [3] Cheng, G. (2006). Higher order semiparametric frequentist inference and the profile sampler. Ph.D. thesis, Dept. Statistics, Univ. Wisconsin-Madison.
  • [4] Cox, D. R. (1972). Regression models and life-tables. J. Roy. Statist. Soc. Ser. B 34 187–220.
  • [5] Dalalyan, A. S., Golubev, G. K. and Tsybakov, A. B. (2006). Penalized maximum likelihood and semiparametric second-order efficiency. Ann. Statist. 34 169–201.
  • [6] Ghosh, J. K. and Mukerjee, R. (1991). Characterization of priors under which Bayesian and frequentist Bartlett corrections are equivalent in the multiparameter case. J. Multivariate Anal. 38 385–393.
  • [7] Groeneboom, P. (1991). Nonparametric maximum likelihood estimators for interval censoring and deconvolution. Technical Report No. 378, Dept. Statistics, Stanford Univ.
  • [8] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • [9] Härdle, W. and Tsybakov, A. B. (1993). How sensitive are average derivatives? J. Econometrics 58 31–48.
  • [10] Huang, J. (1996). Efficient estimation for the Cox model with interval censoring. Ann. Statist. 24 540–568.
  • [11] Johnson, R. A. (1970). Asymptotic expansions associated with posterior distributions. Ann. Math. Statist. 41 851–864.
  • [12] Kalbfleisch, J. D. (1978). Nonparametric Bayesian analysis of survival time data. J. Roy. Statist. Soc. Ser. B 40 214–221.
  • [13] Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference. Springer, New York.
  • [14] Kosorok, M. R., Lee, B. L. and Fine, J. P. (2004). Robust inference for univariate proportional hazards frailty regression models. Ann. Statist. 32 1448–1491.
  • [15] Kuo, H. H. (1975). Gaussian Measure on Banach Spaces. Lecture Notes in Math. 463. Springer, Berlin.
  • [16] Lawley, D. N. (1956). A general method for approximating to the distribution of likelihood ratio criteria. Biometrika 43 295–303.
  • [17] Lee, B. L., Kosorok, M. R. and Fine, J. P. (2005). The profile sampler. J. Amer. Statist. Assoc. 100 960–969.
  • [18] Ma, S. and Kosorok, M. R. (2005). Robust semiparametric M-estimation and the weighted bootstrap. J. Multivariate Analysis 96 190–217.
  • [19] Murphy, S. A., Rossini, A. J. and van der Vaart, A. W. (1997). MLE in the proportional odds model. J. Amer. Statist. Assoc. 92 968–976.
  • [20] Murphy, S. A. and van der Vaart, A. W. (1997). Semiparametric likelihood ratio inference. Ann. Statist. 25 1471–1509.
  • [21] Murphy, S. A. and Van der Vaart, A. W. (1999). Observed information in semiparametric models. Bernoulli 5 381–412.
  • [22] Murphy, S. A. and Van der Vaart, A. W. (2000). On profile likelihood (with discussion). J. Amer. Statist. Assoc. 95 449–485.
  • [23] Murphy, S. A. and Van der Vaart, A. W. (2001). Semiparametric mixtures in case-control studies. J. Multivariate Anal. 79 1–32.
  • [24] Roeder, K., Carroll, R. J. and Lindsay, B. G. (1996). A semiparametric mixture approach to case-control studies with errors in covariables. J. Amer. Statist. Assoc. 91 722–732.
  • [25] Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
  • [26] Shen, X. (2002). Asymptotic normality in semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • [27] van der Vaart, A.W. (1996). New Donsker classes. Ann. Probab. 24 2128–2140.
  • [28] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press.
  • [29] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York.
  • [30] Welch, B. L. and Peers, H. W. (1963). On formulae for confidence points based on integrals of weighted likelihood. J. Roy. Statist. Soc. Ser. B 25 318–329.
  • [31] Wellner, J. A. and Zhang, Y. (2007). Two likelihood-based semiparametric estimation methods for panel count data with covariates. Ann. Statist. 35 2106–2142.
  • [32] Wouk, A. (1979). A Course of Applied Functional Analysis. Wiley, New York.