Annals of Statistics

General frequentist properties of the posterior profile distribution

Guang Cheng and Michael R. Kosorok

Full-text: Open access


In this paper, inference for the parametric component of a semiparametric model based on sampling from the posterior profile distribution is thoroughly investigated from the frequentist viewpoint. The higher-order validity of the profile sampler obtained in Cheng and Kosorok [Ann. Statist. 36 (2008)] is extended to semiparametric models in which the infinite dimensional nuisance parameter may not have a root-n convergence rate. This is a nontrivial extension because it requires a delicate analysis of the entropy of the semiparametric models involved. We find that the accuracy of inferences based on the profile sampler improves as the convergence rate of the nuisance parameter increases. Simulation studies are used to verify this theoretical result. We also establish that an exact frequentist confidence interval obtained by inverting the profile log-likelihood ratio can be estimated with higher-order accuracy by the credible set of the same type obtained from the posterior profile distribution. Our theory is verified for several specific examples.

Article information

Ann. Statist., Volume 36, Number 4 (2008), 1819-1853.

First available in Project Euclid: 16 July 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G20: Asymptotic properties 62F25: Tolerance and confidence regions
Secondary: 62F15: Bayesian inference 62F12: Asymptotic properties of estimators

Semiparametric models Markov chain Monte Carlo profile likelihood higher-order frequentist inference Cox proportional hazards model partly linear regression model


Cheng, Guang; Kosorok, Michael R. General frequentist properties of the posterior profile distribution. Ann. Statist. 36 (2008), no. 4, 1819--1853. doi:10.1214/07-AOS536.

Export citation


  • [1] Bartlett, M. S. (1937). Properties of sufficiency and statistical tests. Proc. Roy. Soc. London Ser. A 160 268–282.
  • [2] Birman, M. S. and Solomjak, M. J. (1967). Piece-wise polynomial approximations of functions of the classes Wpα. Mat. Sbornik 3 295–317.
  • [3] Cheng, G. (2006). Higher order semiparametric frequentist inference and the profile sampler. Ph.D. thesis, Dept. Statistics, Univ. Wisconsin–Madison.
  • [4] Cheng, G. and Kosorok, M. R. (2008). Higher order semiparametric frequentist inference with the profile sampler. Ann. Statist. 36 1786–1818.
  • [5] Cox, D. R. (1972). Regression models and life-tables. J. Roy. Statist. Soc. Ser. B 34 187–220.
  • [6] Dalalyan, A. S., Golubev, G. K. and Tsybakov, A. B. (2006). Penalized maximum likelihood and semiparametric second-order efficiency. Ann. Statist. 34 169–201.
  • [7] Groeneboom, P. (1991). Nonparametric maximum likelihood estimators for interval censoring and deconvolution. Technical Report 378, Dept. Statistics, Stanford Univ.
  • [8] Härdle, W. and Tsybakov, A. B. (1993). How sensitive are average derivatives? J. Econometrics 58 31–48.
  • [9] Huang, J. (1996). Efficient estimation for the Cox model with interval censoring. Ann. Statist. 24 540–568.
  • [10] Kass, R. E. and Wasserman, L. (1996). Formal rules for selecting prior distributions: A review and annotated bibliography. J. Amer. Statist. Assoc. 91 1343–1370.
  • [11] Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference. Springer, New York.
  • [12] Kosorok, M. R., Lee, B. L. and Fine, J. P. (2004). Robust inference for univariate proportional hazards frailty regression models. Ann. Statist. 32 1448–1491.
  • [13] Kuo, H. H. (1975). Gaussian Measure on Banach Spaces. Springer, Berlin.
  • [14] Lee, B. L., Kosorok, M. R. and Fine, J. P. (2005). The profile sampler. J. Amer. Statist. Assoc. 100 960–969.
  • [15] Ma, S. and Kosorok, M. R. (2005). Penalized log-likelihood estimation for partly linear transformation models with current status data. Ann. Statist. 33 2256–2290.
  • [16] Murphy, S. A. and Van der Vaart, A. W. (1997). Semiparametric likelihood ratio inference. Ann. Statist. 25 1471–1509.
  • [17] Murphy, S. A. and Van der Vaart, A. W. (1999). Observed information in semiparametric models. Bernoulli 5 381–412.
  • [18] Murphy, S. A. and Van der Vaart, A. W. (2000). On profile likelihood. J. Amer. Statist. Assoc. 95 1461–1474.
  • [19] Mukerjee, R. and Ghosh, M. (1997). Second-order probability matching priors. Biometrika 84 970–975.
  • [20] Reid, N. (1995). Likelihood and Bayesian approximation methods (with discussion). In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 351–368. Oxford Univ. Press.
  • [21] Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • [22] van de Geer, S. (2000). Empirical Processes in M-estimation. Cambridge Univ. Press.
  • [23] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [24] Welch, B. L. and Peers, H. W. (1963). On formulae for confidence points based on integrals of weighted likelihood. J. Roy. Statist. Soc. Ser. B 25 318–329.