Source: Electron. J. Statist. Volume 3
(2009), 259-289.
When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family. We propose a practical algorithm for finding the MLE in the completion based on repeated linear programming using the R contributed package rcdd and illustrate it with three generalized linear model examples. When the MLE for the null hypothesis lies in the completion, likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need to be adjusted. When the MLE lies in the completion, confidence intervals are changed much more from the usual case. The MLE of the natural parameter can be thought of as having gone to infinity in a certain direction, which we call a generic direction of recession. We propose a new one-sided confidence interval which says how close to infinity the natural parameter may be. This maps to one-sided confidence intervals for mean values showing how close to the boundary of their support they may be.
References
[1] Agresti, A. (2002)., Categorical Data Analysis, second ed. Wiley, Hoboken.
[2] Barndorff-Nielsen, O.E. (1978)., Information and Exponential Families. Wiley, Chichester.
Mathematical Reviews (MathSciNet):
MR489333
[3] Brown, L.D. (1986)., Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory. Institute of Mathematical Statistics, Hayward, CA.
Mathematical Reviews (MathSciNet):
MR882001
[4] Fienberg, S.E., Rinaldo, A. and Zhou, Y. (2009). Maximum Likelihood Estimation in Discrete Exponential Families with Application to Exponential Random Graph Models., Electronic Journal of Statistics (to appear).
[5] Fukuda, K. (2008)., cddlib library, version 094f. http://www.ifor.math. ethz.ch/˜fukuda/cdd_home/cdd.html.
[6] Geyer, C.J. (1990). Likelihood and exponential families. Ph.D. thesis, University of Washington, Seattle. http://www.stat.umn.edu/geyer/, thesis/th.pdf.
[7] Geyer, C.J. (2008a)., R package aster (Aster Models), version 0.7-5. http://www.stat.umn.edu/geyer/aster/.
[8] Geyer, C.J. (2008b). Supporting theory and data analysis for “likelihood inference in exponential families and directions of recession”. Tech. Rep. 672, University of Minnesota School of Statistics., http://www.stat.umn.edu/geyer/gdor/phaseTR.pdf.
[9] Geyer, C.J. (2009). More supporting data analysis for “likelihood inference in exponential families and directions of recession”. Tech. Rep. 673, University of Minnesota School of Statistics. http://www.stat.umn.edu/, geyer/gdor/phase2TR.pdf.
[10] Geyer, C.J. and Meeden, G.D. (2005). Fuzzy and randomized confidence intervals and P-values (with discussion)., Statistical Science 20, 358–387.
[11] Geyer, C.J. and Meeden, G.D. (2008)., R package rcdd (C Double Description for R), version 1.1. Incorporates code from [5]. http://www. stat.umn.edu/geyer/rcdd/.
[12] Geyer, C.J. and Thompson, E.A. (1992). Constrained Monte Carlo maximum likelihood for dependent data (with discussion)., Journal of the Royal Statistical Society, Series B 54, 657–699.
[13] Geyer, C.J., Wagenius, S., and Shaw, R.G. (2007). Aster models for life history analysis., Biometrika 94, 415–426.
[14] Hastie, T., Tibshirani, R., and Friedman, J. (2009)., The Elements of Statistical Learning, second ed. Springer-Verlag, New York.
[15] Lehmann, E.L. (1959)., Testing Statistical Hypotheses. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR107933
[16] Rinaldo, A. (2005). On Maximum Likelihood Estimation For Contingency Tables. Ph.D. Dissertation, Department of Statistics, Carnegie Mellon University, Pittsburgh.
[17] Rockafellar, R.T. (1970)., Convex Analysis. Princeton University Press, Princeton.
Mathematical Reviews (MathSciNet):
MR274683
[18] Rockafellar, R.T. and Wets, R.J.-B. (2004)., Variational Analysis, corrected second printing. Springer-Verlag, Berlin.
[19] Shaw, R.G., Geyer, C.J., Wagenius, S., Hangelbroek, H. H., and Etterson, J. R. (2008). Unifying life history analysis for inference of fitness and population growth., American Naturalist 172, E35–E47.
