The Bayesian approach to statistical problems, though fruitful in many ways, has been rather unsuccessful in treating nonparametric problems. This is due primarily to the difficulty in finding workable prior distributions on the parameter space, which in nonparametric ploblems is taken to be a set of probability distributions on a given sample space. There are two desirable properties of a prior distribution for nonparametric problems. (I) The support of the prior distribution should be large--with respect to some suitable topology on the space of probability distributions on the sample space. (II) Posterior distributions given a sample of observations from the true probability distribution should be manageable analytically. These properties are antagonistic in the sense that one may be obtained at the expense of the other. This paper presents a class of prior distributions, called Dirichlet process priors, broad in the sense of (I), for which (II) is realized, and for which treatment of many nonparametric statistical problems may be carried out, yielding results that are comparable to the classical theory. In Section 2, we review the properties of the Dirichlet distribution needed for the description of the Dirichlet process given in Section 3. Briefly, this process may be described as follows. Let $\mathscr{X}$ be a space and $\mathscr{A}$ a $\sigma$-field of subsets, and let $\alpha$ be a finite non-null measure on $(\mathscr{X}, \mathscr{A})$. Then a stochastic process $P$ indexed by elements $A$ of $\mathscr{A}$, is said to be a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$ if for any measurable partition $(A_1, \cdots, A_k)$ of $\mathscr{X}$, the random vector $(P(A_1), \cdots, P(A_k))$ has a Dirichlet distribution with parameter $(\alpha(A_1), \cdots, \alpha(A_k)). P$ may be considered a random probability measure on $(\mathscr{X}, \mathscr{A})$, The main theorem states that if $P$ is a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$, and if $X_1, \cdots, X_n$ is a sample from $P$, then the posterior distribution of $P$ given $X_1, \cdots, X_n$ is also a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with a parameter $\alpha + \sum^n_1 \delta_{x_i}$, where $\delta_x$ denotes the measure giving mass one to the point $x$. In Section 4, an alternative definition of the Dirichlet process is given. This definition exhibits a version of the Dirichlet process that gives probability one to the set of discrete probability measures on $(\mathscr{X}, \mathscr{A})$. This is in contrast to Dubins and Freedman [2], whose methods for choosing a distribution function on the interval [0, 1] lead with probability one to singular continuous distributions. Methods of choosing a distribution function on [0, 1] that with probability one is absolutely continuous have been described by Kraft [7]. The general method of choosing a distribution function on [0, 1], described in Section 2 of Kraft and van Eeden [10], can of course be used to define the Dirichlet process on [0, 1]. Special mention must be made of the papers of Freedman and Fabius. Freedman [5] defines a notion of tailfree for a distribution on the set of all probability measures on a countable space $\mathscr{X}$. For a tailfree prior, posterior distribution given a sample from the true probability measure may be fairly easily computed. Fabius [3] extends the notion of tailfree to the case where $\mathscr{X}$ is the unit interval [0, 1], but it is clear his extension may be made to cover quite general $\mathscr{X}$. With such an extension, the Dirichlet process would be a special case of a tailfree distribution for which the posterior distribution has a particularly simple form. There are disadvantages to the fact that $P$ chosen by a Dirichlet process is discrete with probability one. These appear mainly because in sampling from a $P$ chosen by a Dirichlet process, we expect eventually to see one observation exactly equal to another. For example, consider the goodness-of-fit problem of testing the hypothesis $H_0$ that a distribution on the interval [0, 1] is uniform. If on the alternative hypothesis we place a Dirichlet process prior with parameter $\alpha$ itself a uniform measure on [0, 1], and if we are given a sample of size $n \geqq 2$, the only nontrivial nonrandomized Bayes rule is to reject $H_0$ if and only if two or more of the observations are exactly equal. This is really a test of the hypothesis that a distribution is continuous against the hypothesis that it is discrete. Thus, there is still a need for a prior that chooses a continuous distribution with probability one and yet satisfies properties (I) and (II). Some applications in which the possible doubling up of the values of the observations plays no essential role are presented in Section 5. These include the estimation of a distribution function, of a mean, of quantiles, of a variance and of a covariance. A two-sample problem is considered in which the Mann-Whitney statistic, equivalent to the rank-sum statistic, appears naturally. A decision theoretic upper tolerance limit for a quantile is also treated. Finally, a hypothesis testing problem concerning a quantile is shown to yield the sign test. In each of these problems, useful ways of combining prior information with the statistical observations appear. Other applications exist. In his Ph. D. dissertation [1], Charles Antoniak finds a need to consider mixtures of Dirichlet processes. He treats several problems, including the estimation of a mixing distribution, bio-assay, empirical Bayes problems, and discrimination problems.

We show that, in a weak sense, as the cost of observation tends to zero, the shape of the continuation region of the Bayes solution for the exponential family problem given above is approximated by that of a corresponding problem for the Wiener process with drift. The approach is an extension of that used in Bickel and Yahav, Proc. Sixth Berkeley Symp. Math. Statist. Prob. (1971).

A study of relationships between confidence regions being Bayesian, and the existence of some generalizations of Fisher's notion of relevant subsets. For a betting scheme introduced by Buehler, and for finite parameter space, it is shown that non-Bayesian procedures allow a winning strategy for a statistician's adversary. It is further shown, for finite parameter space, non-Bayesian procedures must admit conditional confidence levels bounded away from the unconditional level, the converse to a theorem of Wallace. For general parameter space these results follow from a procedure not being weak Bayes in a certain sense.

Robust test problems between two approximately known simple hypotheses can be formalized as minimax test problems between two composite hypotheses. We show that if the composite hypotheses can be described in terms of alternating capacities of order 2 (in the sense of Choquet), then the minimax tests are ordinary Neyman-Pearson tests between a fixed representative pair of simple hypotheses; moreover, the condition is in a certain sense also necessary. All the neighborhoods customarily used to formalized approximate knowledge happen to have this particular structure.

Let $X_1,\cdots, X_n$ be i.i.d. random variables each with a density which is $a^{-1} \exp (b - x)$ or 0 according as $x \geqq b$ or $x < b$ where $-\infty < b < \infty, a > 0$ are unknown constants. Let $\bar{X} = n^{-1} \sum X_i$ and $M = \min X_i$. The maximum likelihood estimator of $a$ is $\bar{X} - M$ and is, if quadratic loss is assumed, the best affine equivariant estimator of $a$. It is shown that if loss is measured by any member of a large class of "bowl-shaped" functions which includes quadratic loss, the best affine equivariant estimator is inadmissible. The proof entails an examination of the conditional expected loss given the maximal invariant under the scale group. It is carried out by exhibiting a superior alternative. In the case of quadratic loss, for example, the result is as follows. Given any estimator $u = (\bar{X} - M)T\lbrack M(\bar{X} - M)^{-1}\rbrack$, let $T^\ast(y) = T(y)$ if $y < 0$ and $T^\ast(y) = \min \{T(y), n(n + 1)^{-1}(1 + y)\}$ if $y > 0$. If $T^\ast \neq T$ with positive probability, then the estimator, obtained from $u$ by replacing $T$ by $T^\ast$, has uniformly smaller risk than $u$. Using a generalization of the author's conditions for admissibility [Ann. Math. Statist. 41 (1970) 446-457] a class, $B$, of generalized Bayes estimators within $D$, the class of scale equivariant estimators, are obtained with each member of $B$ admissible in $D$. The improper measures determining members of $B$ have densities on the orbit space $R$, created in the parameter space by the action of the group of scale changes. These prior densities, $g$, satisfy $\int^\infty_1 (t^2g(t))^{-1} dt = \int^{-1}_{-\infty} (t^2g(t))^{-1} dt = \infty$.

The weak convergence of the sample df is studied under a given sequence of alternative hypotheses when parameters are estimated from the data. For a general class of estimators it is shown that the sample df, when normalised, converges weakly to a specified normal process. The results are specialised to the case of efficient estimation.

The permutation test using the usual $F$-statistic from a randomized block experiment is considered under a randomization model. Alternative hypotheses assuming additive treatment effects are considered. It is shown that the critical value of the test statistic tends to a constant in probability as the number of blocks becomes large. The large-sample power of the test is calculated for a sequence of alternatives arising naturally from the randomization model.

For a sequence of random variables forming an $m$-dependent stochastic process (not necessarily stationary), asymptotic distribution and other convergence properties of the extremum of certain functions of the empirical distribution are studied. In this context, it is shown that the asymptotic probability of the classical Kolmogorov-Smirnov statistic exceeding any positive real number provides an upper bound for the corresponding probability when the underlying random variables are not necessarily identically distributed. The theory is specifically applied to the study of the limiting distribution, strong convergence and convergence of the first moment of the strength of a bundle of parallel filaments (which is shown to be the extremum of a function of the empirical distribution).

Consider a multiple regression problem in which the dependent variable and (3 or more) independent variables have a joint normal distribution. This problem was investigated some time ago by Charles Stein, who proposed reasonable loss functions for various problems involving estimation of the regression coefficients and who obtained various minimax and admissibility results. In this paper we continue this investigation and establish the inadmissibility of the traditional maximum likelihood estimators. Inadmissibility is proved by exhibiting explicit procedures having lower risk than the corresponding maximum likelihood procedure. These results are given in Theorems 1 and 2 of Section 3.

Let $\{X(t), t = 0, \pm 1, \pm 2,\cdots\}$ be a $p$-dimensional, zero mean, stationary Gaussian time series with matrix-valued covariance function $R(u) = (r_{jk}(u)), u = 0, \pm 1, \pm 2, \cdots; j, k = 1, 2,\cdots, p$. Let $F(\omega)$ be the spectral density matrix of the time series (if $F(\omega)$ exists), and assume that $F(\omega)$ is positive definite for all $\omega\in(0, 2\pi\rbrack$. Let $\hat{F}_X(\omega_l)$ be an estimator of $F(\omega_l)$ formed by averaging $(2n + 1)$ periodogram ordinates centered and equally spaced around $\omega_l, l = 1, 2,\cdots, M$, where the $\omega_l$'s themselves are equally spaced on $(0, \pi\rbrack$, and where all periodogram ordinates are based on the same record of length $T, (2n + 1)M \leqq T/2$, taken from the time series $\{X(t)\}$. Wahba (1968) has shown that if $\sum_j\sum_k\sum_u|ur_{jk}(u)| < \infty$ and $\log_2 M \leqq n$, then it is possible to construct $M$ independent complex Wishart matrices $W_T(\omega_l), l = 1, 2,\cdots, M$, such that $\{\hat{F}_X(\omega_l), l = 1, 2,\cdots, M\}$ converge simultaneously in mean square to $\{W_T(\omega_l), l = 1, 2,\cdots, M\}$ as $n, M$ (and thus $T$) get large. In the present paper, it is shown that Wahba's result holds under the less restrictive condition that $\sum_j\sum_k\sum_u|u|^{\frac{1}{2}}|r_{jk}(u)| < \infty$, and without our needing to assume that $\log_2 M \leqq n$. In consequence, a form of weak convergence of the averaged periodogram to a certain matrix-valued Wishart stochastic process is demonstrated (something, by the way, that Wahba (1968) cannot show because of her restriction that $\log_2 M \leqq n$). This result is a consequence of some general conclusions concerning the approximation of circulant quadratic forms in the time series $\{X(t)\}$.

We recast a class of denumerable-state, infinite-action Markov renewal programs with unknown parameters as one-state programs with actions corresponding to stationary policies in the original program. Under suitable conditions we find an adaptive (nonstationary) optimal policy in the sense of maximizing long-run expected reward per unit time.

Fedorov (Theory of Optimal Experiments (1972)) gives a sequence of designs converging to a $D$-optimal design. Several modifications of that sequence are given to improve the speed of convergence. The analogous sequence for estimating some of the parameters is shown to converge to a $D$-optimal design, whether or not all the parameters are estimable under the limiting design. We prove the result $d(x, \xi)\xi(x) \leqq 1$, and several related results.

The Polya urn scheme is extended by allowing a continuum of colors. For the extended scheme, the distribution of colors after $n$ draws is shown to converge as $n \rightarrow \infty$ to a limiting discrete distribution $\mu^\ast$. The distribution of $\mu^\ast$ is shown to be one introduced by Ferguson and, given $\mu^\ast$, the colors drawn from the urn are shown to be independent with distribution $\mu^\ast$.

Consider the usual decision theoretic situation where one observes a random vector $X$ from which an estimate of its classification $\theta \in \{0, 1\}$ is to be made. If one knows the a priori probabilities for $\theta$ and the conditional densities of $X$ given $\theta$ then the smallest probability of error which can be achieved is called the Bayes risk and denoted by $R^\ast$. Assuming that the a priori probabilities and conditional densities are unknown we consider the problem of estimating $R^\ast$ from the independent observations $(X_1, \theta_1),\cdots, (X_n, \theta_n)$. Suppose $X$ has an unknown classification $\theta$ where $(X, \theta)$ is independent of the observations $(X_1, \theta_1),\cdots, (X_n, \theta_n)$. If $\{\delta_n\}$ is a sequence of decision procedures, where $\delta_n$ determines the estimate of $\theta$ from $X$ and $(X_1, \theta_1),\cdots, (X_n, \theta_n)$, then the notion of a deleted estimate of $R^\ast$ with $\delta_n$ is introduced and, under mild assumptions, is shown to be a consistent estimate of $R^\ast$.

It will be shown that if a confidence procedure is admissible when the criterion is probability of not covering the true value and expected length (or something more general than length), then the same confidence procedure is at least almost admissible when the criterion is probability of not covering the true value and probability of covering false values. The result is true (under mild conditions) for virtually all confidence region estimation problems.

For a density function $f(x)$, the tail area $\alpha(x) = \int^\infty_x f(x) dx,$ may be approximated by $\hat{\alpha}(x) = \frac{f(x)}{g(x)} \cdot (K - 1)^{-1}\cdot\big\{1 + \frac{1}{2} \big(\frac{g'(x)}{g^2(x)} - (K)\big)\big\},$ where $g(x) = f(x)/f'(x)$, and $K = \lim_{x\rightarrow\infty} \{g'(x)/g^2(x)\}$. The formula requires only one constant and three function evaluations; $g$ and $g'$ are typically elementary functions. Such approximations are useful for programmed calculators or very small computers where only a few constants can be stored. The accuracy of the approximation is calculated for some common distributions. The approximation is very accurate for a large class of distributions.

A model for Bernoulli trials with Markov dependence is developed which possesses the usual frequency parameter $p = P\lbrack X_i = 1\rbrack$ and an additional dependence parameter $\lambda = P\lbrack X_i = 1 \mid X_{i-1} = 1\rbrack$. Sufficient statistics for the model with $p$ and $\lambda$ unknown are found and an exact closed form expression for their small sample joint distribution is given. Large sample distribution theory is also given and small sample variances compared with large sample approximations. Easily computed estimators of $p$ and $\lambda$ are recommended and shown to be asymptotically efficient. With $p$ unknown the u.m.p. unbiased test of independence is noted to be the run test. An application to a rainfall example is given.

Let $Z_1, Z_2,\cdots$, be independent and identically distributed random variables and $\{c_{ijn}\}$ real numbers; put $T_n = \sum^n_{i,j = 1} c_{ijn}Z_iZ_j$. This paper gives conditions under which the distribution of $T_n - ET_n$ converges to the distribution of $\sum \Upsilon_m(Y_m^2 - 1)$ with $\{\Upsilon_m\}$ a real sequence and $Y_1, Y_2,\cdots$ independent $N(0, 1)$ random variables. The results are applied to the calculation of the asymptotic distributions of test criteria of the form $Q_n^W = \sum \lbrack F_0(X_{kn}) - k/n + 1\rbrack^2W(k/n + 1)$ for testing the hypothesis that $X_{1n}, X_{2n},\cdots, X_{nn}$ are the order statistics of an independent sample from the distribution function $F_0$; here $W$ is a weight function.

In this paper we give further reduction for a general MANOVA problem, and show that the general MANOVA problem can be treated as the MANOVA problem with the conditional setup. The last result is used to prove that the power functions of some tests in the general MANOVA models are monotone increasing functions of each of the noncentrality parameters, and, in particular, the likelihood-ratio test has the monotonicity property.

Let $X_{(i\mid n)}$ be the $i$th smallest order statistic from a population with $\operatorname{pdf} f(x)$ and $\operatorname{cdf} F(x)$. When $\tilde{x}$ is the population median, $n$ is the sample size and $G(x) = F(x)(1 - F(x))$, the following are proved. \begin{equation*}\begin{split}E(X_{(n-i+1\mid n)}) + E(X_{(i\mid n)}) \\ = n(n - 2i + 1)\binom{n - 1}{i - 1} \sum^{\lbrack (n-2i+1)/2\rbrack}_{\nu = 0}\frac{(-1)^\nu}{n - 2i + 1 - \nu}\binom{n - 2i + 1 - \nu}{\nu} \\ \times \Psi_1(i - 1 + \nu),\end{split}\end{equation*} $E(X_{(n-i+1\mid n)}) - E(X_{(i\mid n)}) = n\binom{n - 1}{i - 1} \sum^{\lbrack (n-2i)/2\rbrack}_{\nu = 0}(-1)^\nu\binom{n-2i-\nu}{\nu}\Psi_2(i - 1 + \nu)$ for $i = 1,2,\cdots,\lbrack n/2 \rbrack$ and $E(X_{(i\mid n)}) = n\binom{n - 1}{i - 1}\Psi_1(i - 1)$ for $i = (n + 1)/2$ when $n$ is odd, where $\Psi_1(l) = \int^\infty_{-\infty} x f(x)\{G(x)\}^l dx$ and \begin{equation*}\begin{split}\Psi_2(l) = -\int^{\tilde{x}}_{-\infty} x f(x) \{G(x)\}^l(1 - 4G(x))^{\frac{1}{2}} dx \\ + \int^\infty_{\tilde{x}}x f(x) \{G(x)\}^l(1 - 4G(x))^{\frac{1}{2}} dx.\end{split}\end{equation*} Parallel formulas are obtained for the $k$th order moment.

The minimum variance unbiased estimates are obtained when a sample is drawn from the power series distribution with unknown parameter $\lambda$ and unknown truncation parameter $\nu$. Unbiased tests of hypotheses are formulated.