From Unit Root to Stein’s Estimator to Fisher’s k Statistics: If You Have a Moment, I Can Tell You More

Abstract

Any general textbook that discusses moment generating functions (MGFs) shows how to obtain a moment of positive-integer order via differentiation, although usually the presented examples are only illustrative, because the corresponding moments can be calculated in more direct ways. It is thus somewhat unfortunate that very few textbooks discuss the use of MGFs when it becomes the simplest, and sometimes the only, approach for analytic calculation and manipulation of moments. Such situations arise when we need to evaluate the moments of ratios and logarithms, two of the most common transformations in statistics. Such moments can be obtained by differentiating and integrating a joint MGF of the underlying untransformed random variables in appropriate ways. These techniques are examples of multivariate Laplace transform methods and can also be derived from the fact that moments of negative orders can be obtained by integrating an MGF. This article reviews, extends and corrects various results scattered in the literature on this joint-MGF approach, and provides four applications of independent interest to demonstrate its power and beauty. The first application, which motivated this article, is for the exact calculation of the moments of a well-known limiting distribution under the unit-root AR(1) model. The second, which builds on Stigler’s Galtonian perspective, reveals a straightforward, non-Bayesian constructive derivation of the Stein estimator, as well as convenient expressions for studying its risk and bias. The third finds an exceedingly simple bound for the bias of a sample correlation from a bivariate normal population, namely the magnitude of the relative bias is not just of order n⁻¹, but actually is bounded above by n⁻¹ for all sample sizes n≥2. The fourth tackles the otherwise intractable problem of studying the finite-sample optimal bridge in the context of bridge sampling for computing normalizing constants. A by-product of the joint-MGF approach is that positive-order fractional moments can be easily obtained from an MGF without invoking the concept of fractional differentiation, a method used by R. A. Fisher in his study of k statistics 45 years before it reappeared in the probability literature.