The Annals of Mathematical Statistics

Transformations Related to the Angular and the Square Root

Abstract

The use of transformations to stabilize the variance of binomial or Poisson data is familiar(Anscombe , Bartlett [2, 3], Curtiss , Eisenhart ). The comparison of transformed binomial or Poisson data with percentage points of the normal distribution to make approximate significance tests or to set approximate confidence intervals is less familiar. Mosteller and Tukey  have recently made a graphical application of a transformation related to the square-root transformation for such purposes, where the use of "binomial probability paper" avoids all computation. We report here on an empirical study of a number of approximations, some intended for significance and confidence work and others for variance stabilization. For significance testing and the setting of confidence limits, we should like to use the normal deviate $K$ exceeded with the same probability as the number of successes $x$ from $n$ in a binomial distribution with expectation $np$, which is defined by $\frac{1}{2\pi} \int^K_{-\infty} e^{-\frac{1}{2}t^2} dt = \operatorname{Prob} \{x \leq k |mid \operatorname{binomial}, n, p\}.$ The most useful approximations to $K$ that we can propose here are $N$ (very simple), $N^+$ (accurate near the usual percentage points), and $N^{\ast\ast}$ (quite accurate generally), where $N = 2 (\sqrt{(k + 1)q} - \sqrt{(n - k)p)}.$ (This is the approximation used with binomial probability paper.) $N^+ = N + \frac{N + 2p - 1}{12\sqrt{E}},\quad E = \text{lesser of} np \text{and} nq, N^\ast = N + \frac{(N - 2)(N + 2)}{12} \big(\frac{1}{\sqrt{np + 1}} - \frac{1}{\sqrt{nq + 1}}\big), N^{\ast\ast} = N^\ast + \frac{N^\ast + 2p - 1}{12 \sqrt{E}}\cdot\quad E = \text{lesser of} np \text{and} nq.$ For variance stabilization, the averaged angular transformation $\sin^{-1}\sqrt{\frac{x}{n + 1}} + \sin^{-1} \sqrt{\frac{x + 1}{n+1}}$ has variance within $\pm 6%$ of $\frac{1}{n + \frac{1}{2}} \text{(angles in radians)}, \frac{821}{n + \frac{1}{2}} \text{(angles in degrees)},$ for almost all cases where $np \geq 1$. In the Poisson case, this simplifies to using $\sqrt{x} + \sqrt{x + 1}$ as having variance 1.

Article information

Source
Ann. Math. Statist., Volume 21, Number 4 (1950), 607-611.

Dates
First available in Project Euclid: 28 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoms/1177729756

Digital Object Identifier
doi:10.1214/aoms/1177729756

Mathematical Reviews number (MathSciNet)
MR38028

Zentralblatt MATH identifier
0039.35304

JSTOR