Statistical Science

Estimating Random Effects via Adjustment for Density Maximization

Carl Morris and Ruoxi Tang

Full-text: Open access


We develop and evaluate point and interval estimates for the random effects θi, having made observations yi|θiind N[θi, Vi], i = 1, …, k that follow a two-level Normal hierarchical model. Fitting this model requires assessing the Level-2 variance A ≡ Var(θi) to estimate shrinkages BiVi / (Vi + A) toward a (possibly estimated) subspace, with Bi as the target because the conditional means and variances of θi depend linearly on Bi, not on A. Adjustment for density maximization, ADM, can do the fitting for any smooth prior on A. Like the MLE, ADM bases inferences on two derivatives, but ADM can approximate with any Pearson family, with Beta distributions being appropriate because shrinkage factors satisfy 0 ≤ Bi ≤ 1.

Our emphasis is on frequency properties, which leads to adopting a uniform prior on A ≥ 0, which then puts Stein’s harmonic prior (SHP) on the k random effects. It is known for the “equal variances case” V1 = ⋯ = Vk that formal Bayes procedures for this prior produce admissible minimax estimates of the random effects, and that the posterior variances are large enough to provide confidence intervals that meet their nominal coverages. Similar results are seen to hold for our approximating “ADM-SHP” procedure for equal variances and also for the unequal variances situations checked here.

For shrinkage coefficient estimation, the ADM-SHP procedure allows an alternative frequency interpretation. Writing L(A) as the likelihood of Bi with i fixed, ADM-SHP estimates Bi as B̂i = Vi / (Vi + Â) with  ≡ argmax (AL(A)). This justifies the term “adjustment for likelihood maximization,” ALM.

Article information

Statist. Sci., Volume 26, Number 2 (2011), 271-287.

First available in Project Euclid: 1 August 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Shrinkage ADM Normal multilevel model Stein estimation objective Bayes


Morris, Carl; Tang, Ruoxi. Estimating Random Effects via Adjustment for Density Maximization. Statist. Sci. 26 (2011), no. 2, 271--287. doi:10.1214/10-STS349.

Export citation


  • Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions. Applied Mathematics Series 55. National Bureau of Standards, Washington, DC.
  • Bell, W. (1999). Accounting for uncertainty about variances in small area estimation. Available at did/www/saipe/publications/conference.html.
  • Christiansen, C. L. and Morris, C. N. (1997). Hierarchical Poisson regression modeling. J. Amer. Statist. Assoc. 92 618–632.
  • Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1–38.
  • Efron, B. and Morris, C. N. (1975). Data analysis using Stein’s estimator and it’s generalizations. J. Amer. Statist. Assoc. 70 311–319.
  • Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC Press, Boca Raton, FL.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. Probab. I 361–379. Univ. California Press, Berkeley.
  • Li, H. and Lahiri, P. (2010). An adjusted maximum likelihood method for solving small area estimation problems. J. Multivariate Anal. 101 882–892.
  • Morris, C. N. (1977). Inverval estimation for empirical Bayes generalizations of Stein’s estimator. In Proceedings of the Twenty-Second Conference on the Design of Experiments in Army Research Development and Testing. ARO Report 77-2.
  • Morris, C. N. (1983a). Parametric empirical Bayes confidence intervals. In Scientific Inference, Data Analysis, and Robustness (Madison, Wis., 1981). Publ. Math. Res. Center Univ. Wisconsin 48 ( G. E. Box, T. Leonard and C.-F. Wu, eds.) 25–50. Academic Press, Orlando, FL.
  • Morris, C. N. (1983b). Parametric empirical Bayes inference: Theory and applications (with discussion). J. Amer. Statist. Assoc. 78 47–65.
  • Morris, C. (1988a). Determining the accuracy of Bayesian empirical Bayes estimates in the familiar exponential families. In Statistical Decision Theory and Related Topics IV 1 (West Lafayette, Ind., 1986) ( S. Gupta and J. Berger, eds.) 251–263. Springer, New York.
  • Morris, C. N. (1988b). Approximating posterior distributions and posterior moments. In Bayesian Statistics 3 ( J.-M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 327–344. Oxford Univ. Press, New York.
  • Rasbash, J., Browne, W., Goldstein, H., Yang, M., Plewis, I., Healy, M., Woodhouse, G., Draper, D., Langford, I. and Lewis, T. (2001). A User’s Guide to MLwiN. Centre for Multilevel Modelling. Institute of Education, Univ. London.
  • Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • Tang, R. (2002). Fitting and evaluating certain two-level hierarchical models. Ph.D. thesis, Harvard Univ.

See also

  • Discussion of: Estimating Random Effects via Adjustment for Density Maximization by C. Morris and R. Tang.
  • Discussion of: Estimating Random Effects via Adjustment for Density Maximization by C. Morris and R. Tang.
  • Rejoinder: Estimating Random Effects via Adjustment for Density Maximization.