Let $N$ be the stopping variable of a SPRT for testing one composite hypothesis against another, based on i.i.d. observations $Z_1, Z_2, \cdots$ with common distribution $P. P$ need not belong to the model. $N$ is termed exponentially bounded if for every choice of stopping bounds there exists $c < \infty$ and $\rho < 1$ such that $P\{N > n\} < c\rho^n$; if this does not hold $P$ is called obstructive. The main theorem presents sufficient conditions, both on the model and on $P$, for $N$ to be exponentially bounded. Under weaker conditions the theorem proves $P\{N < \infty\} = 1$. Two applications of the theorem are given: 1. In the problem of testing $\sigma = \sigma_1$ against $\sigma = \sigma_0$ in a normal population with unknown mean it is proved that $N$ is exponentially bounded for every $P$ except if $P\{Z_1 = \zeta \pm a\} = \frac{1}{2} (\zeta$ arbitrary and $a^2$ a given function of $\sigma_1$ and $\sigma_2)$ in which case $P$ is obstructive. 2. In the sequential $t$-test it is proved that $N$ is exponentially bounded for every $P$ for which $Z_1^2$ has finite $\operatorname{mgf}$ and is not a member of a certain family of two-point distributions.

An algorithm for density estimation based on ordinary polynomial (Lagrange) interpolation is studied. Let $F_n(x)$ be $n/(n + 1)$ times the sample c.d.f. based on $n$ order statistics, $t_1, t_2, \cdots t_n$, from a population with density $f(x)$. It is assumed that $f^{(v)}$ is continuous, $v = 0, 1, 2,\cdots, r, r = m - 1$, and $f^{(m)} \in L_2(-\infty, \infty). F_n(x)$ is first locally interpolated by the $m$th degree polynomial passing through $F_n(t_{ik_n}), F_n(t_{(i+1)k_n}),\cdots F_n(t_{(i+m)k_n})$, where $k_n$ is a suitably chosen number, depending on $n$. The density estimate is then, locally, the derivative of this interpolating polynomial. If $k_n = O(n^{(2m-1)/(2m)})$, then it is shown that the mean square convergence rate of the estimate to the true density is $O(n^{-(2m-1)/(2m)})$. Thus these convergence rates are slightly better than those obtained by the Parzen kernel-type estimates for densities with $r$ continuous derivatives. If it is assumed that $f^{(m)}$ is bounded, and $k_n = O(n^{2m/(2m+1)})$, then it is shown that the mean square convergence rates are $O(n^{-2m/(2m+1)})$, which are the same as those of the Parzen estimates for $m$ continuous derivatives. An interesting theorem about Lagrange interpolation, concerning how well a function can be interpolated knowing only its integral at nearby points, is also demonstrated.

Two very closely related definitions of robustness of a sequence of estimators are given which take into account the types of deviations from parametric models that occur in practice. These definitions utilize the properties of the Prokhorov distance between probability distributions. It is proved that weak $^\ast$-continuous functionals on the space of probability distributions define robust sequences of estimators (in either sense). The concept of the "breakdown point" of a sequence of estimators is defined, and some examples are given.

A problem of optimal stopping is formulated and simple rules are proposed which are asymptotically optimal in an appropriate sense. The problem is of central importance in quality control and also has applications in reliability theory and other areas.

Let $X_1, \cdots, X_n$ be a sample from a unimodal distribution, $F$, and let $\{a_n\}$ be a sequence converging to zero. A nonparametric estimate of the mode is the center of the interval of length $2a_n$ containing the most observations. This estimate is shown to be strongly consistent and conditions on the speed at which $a_n$ may converge to zero are given. This estimator of the mode is related to the naive density estimator, $(F_n(x + a_n) - F_n(x - a_n))/2a_n$, where $F_n$ is the empirical distribution function. A simple strong consistency result for this naive density estimator is given. Also other estimators of the mode are discussed briefly and an application of estimators of the mode is mentioned.

A probability forecaster is asked to give a density $p$ of a random variable $\omega$. In return he gets a reward (or score) depending on $p$ and on a subsequently observed value of $\omega$. A scoring rule is called proper if the expected score is maximized when the true density is chosen. The present paper uses convex analysis to generalize McCarthy's characterization of proper scoring rules.

Let $X = \cdot b_1b_2b_3 \cdots$ be a random variable with independent binary digits $b_n$ taking values 0 or 1 with probability $p_n$ and $q_n = 1 - p_n$. When does $X$ have a density? A continuous density? A singular distribution? This note gives necessary and sufficient conditions for the distribution of $X$ to be: discrete: $\Sigma\min (p_n, q_n) < \infty$; singular: $\Sigma^\infty_m\lbrack\log (p_n/q_n)\rbrack^2 = \infty$ for every $m$; absolutely continuous: $\Sigma^\infty_m\lbrack\log (p_n/q_n)\rbrack^2 < \infty$ for some $m$. Furthermore, $X$ has a density that is bounded away from zero on some interval if and only if $\log (p_n/q_n)$ is a geometric sequence with ratio $\frac{1}{2}$ for $n > k$, and in that case the fractional part of $2^k X$ has an exponential density (increasing or decreasing with the uniform a special case).

We introduce a sequential competitive decision process that is a generalization of noncooperative finite games and of two-person zero-sum stochastic games (hence, of Markovian decision processes). We prove the existence of equilibrium points under criteria of discounted gain and of average gain. Two person zero-sum stochastic games and noncooperative finite games were introduced in elegant papers by Shapley [22] and Nash [16], [17]. Shapley's work prompted a series of papers [1], [4], [5], [10], [11], [12], [14], [18], [26] concerned with the existence of minimax solutions and algorithms for their computation. Even for the two-person zero-sum case, no finite algorithm yet exists. Nash's papers led to a sizeable literature in both mathematics and economics. Mills' [15] work, for example, is related to our characterization of equilibrium points in Section 4. Noncooperative stochastic games may yield fruitful models for several phenomena in the social sciences. Theories of economic markets, for example, have increasingly sought to encompass sequential economic decision processes. Some recent research in social psychology has taken an analogous direction [19], [25]. I became aware of recent work by Rogers [20] shortly after completing this paper. His results and ours nearly coincide with our Theorem 2 being slightly stronger than the comparable results in his paper. The basic difference between the papers is that Rogers relies on the Kakutani fixed point theorem whereas we use Brouwer's theorem. Our arguments are somewhat simpler as a consequence.

Let $X_1, X_2, \cdots$ be i.i.d. random variables with mean 0, and let $S_n = \sum^n_{i=1} X_i$. The $S_n/n$ optimal stopping problem is to maximize $E(S_\tau/\tau)$ among finite-valued stopping times $\tau$ relative to the process $(S_n, n \geqq 1)$. In this paper we prove partially Dvoretzky's (1967) conjecture that an optimal stopping time should exist when $E|X_1|^\beta < \infty$ for some $\beta > 1$, by showing that the result holds if $\lim \sup_{n\rightarrow\infty} P(S_n \geqq c\|S_n\|) > 0$ for some $c > 0$, where $\|S_n\| = (E|S_n|^\beta)^{1/\beta}$. This condition is shown to hold in some special cases, including the case where the $X_i$ are in the domain of attraction of a stable distribution with exponent greater than one.

Upper and lower probabilities needed for distribution-free inference about the parameters are found for a class of statistical models. The results improve upon those of a previous paper in that continuity of the underlying distribution function is not assumed.

Let $\{X_i\}$ be a sequence of independent random variables with the same distribution function $F(x) = \mathrm{P r}\{X_i \leqq x\}$. Let $F(M_p) = p, 0 < p < 1$. Suppose $F$ has two derivatives in a neighborhood of $M_p, F"(x)$ is bounded there and $F'(M_p)$ is positive. Let $Y_{p,n}$ be a sample $p$-quantile based on $X_1, \cdots, X_n$. Let $nG_n(x)$ be the number of $X_i$'s among $(X_1, \cdots, X_n)$ which are $> x$. Bahadur (1966) has proved \begin{equation*}\tag{1}Y_{p,n} = M_p + \lbrack G_n(M_p) - (1 - p)\rbrack/F'(M_p) + R_n\end{equation*} where the remainder term $R_n$ becomes negligible as $n \rightarrow \infty$. More precisely, he has shown $R_n = O(n^{-\frac{3}{4}} \log n)$ a.s. as $n \rightarrow \infty$. The best result of this type is due to Kiefer (1967) who has calculated the exact order of $R_n$. Sen (1968) has extended Bahadur's result to random variables which are neither independent nor identically distributed. We shall give a new and much simpler proof of a weaker version of Bahadur's result which suffices for many statistical applications. Our proof involves fewer assumptions than Bahadur's. For arbitrary $p_n$ let $M_{p_n}$ be defined as $M_p + (p_n - p)/F'(M_p)$. Consider \begin{equation*}\tag{2}Y_{p_n,n} = M_{p_n} + \lbrack G_n(M_p) - (1 - p)\rbrack/F'(M_p) + R_n\end{equation*} where $Y_{p_n,n}$ is a sample $p_n$-quantile. In Section 2 we have proved the following result about $R_n$. THEOREM 1. Suppose $F'(M_p)$ exists and is strictly positive and $p_n - p = O(1/n^{\frac{1}{2}})$. Then $R_n$ as defined in (2) (and, a fortiori, $R_n$ as defined in (1) satisfies \begin{equation*}\tag{3}n^{\frac{1}{2}} R_n \rightarrow 0 \text{in probability}.\end{equation*} (After writing this paper the author discovered that the result for $p_n = p$ is stated without proof in Chernoff et al (1967).) It is easy to extend this result as in Sen (1968). An outline is sketched in one of the remarks. Once again it is possible to achieve some economy in assumptions. The representation (1) is not new. Its use in deriving the asymptotic moments of $Y_{p,n}$ goes back to Karl Pearson. See, for example, (1) in Hojo (1931). But the formulation therein is very imprecise and lacks a rigorous justification. We next consider an application of Theorem 1. Let $\bar{X}_n = (\sum^n_1 X_i)/n$ and $P_n =$ proportion of $X_i$'s above $\bar{X}_n$. David (1962) proved the asymptotic normality of $P_n$ when $F$ is a normal distribution function. Using the same elegant trick, Mustafi (1968) has proved a similar result for bivariate normal distributions. We shall extend these results considerably by providing alternative proofs based on Theorem 1, which dispense with the normality assumption on F. Moreover, in our proof we may consider--though we shall not do so for purposes of simplicity--instead of the sample mean $\bar{X}_n$ an $U$-statistic to which the central limit theorem of Hoeffding (1948) applies.

This paper studies the problem of testing the independence of two random variables $X, Y$ from a random sample, $(X_1, Y_1), (X_2, Y_2) \cdots (X_n, Y_n)$, of size $n$, where $X$ and $Y$ are angular variates (i.e., reals modulo 1). In the standard case where $X$ and $Y$ are ordinary real variables, the following approach has been useful. Suppose $(X, Y)$ has the continuous distribution function $F(x, y)$ with marginal distribution functions $F_1(x)$ and $F_2(y)$, respectively. It is desired to test $H_0: F(x, y) = F_1(x)F_2(y)$ against the alternative $H_A: F(x, y) \neq F_1(x)F_2(y)$. Let $F_n(x, y)$ denote the sample distribution function of the random bivariate sample, i.e., if $H(x)$ denotes the left continuous Heaviside function then \begin{equation*}\tag{1}F_n(x, y) = \frac{1}{n} \sum^n_{j=1} H(x - X_j)H(y - Y_j).\end{equation*} Also, let $F_{n1}(x)$ and $F_{n2}(y)$ denote the sample distribution functions associated with the first and second components of the random sample vector. In terms of $H(x)$ \begin{equation*}\tag{2}F_{n1}(x) = \frac{1}{n} \sum^n_{j=1} H(x - X_j)\end{equation*} and \begin{equation*}\tag{3}F_{n2}(y) = \frac{1}{n} \sum^n_{j=1} H(y - Y_j).\end{equation*} Blum, Kiefer and Rosenblatt [1] studied the following distribution free tests of independence based on the sample distribution function. Reject for large values of \begin{equation*}\tag{4}A_n = \sup_{x,y}|T_n(x, y)|\end{equation*} or \begin{equation*}\tag{5}B_n = n \int\int\lbrack T_n(x, y)\rbrack^2 dF_n(x, y),\end{equation*} where \begin{equation*}\tag{6}T_n(x, y) = F_n(x, y) - F_{n1}(x)F_{n2}(y).\end{equation*} The first statistic, constructed in the spirit of Kolomorov-Smirnov statistics, has good power properties, cf. [1], Section 4, but its asymptotic distribution is unknown. The statistic $B_n$ is analogous to the Cramer-von Mises statistic and is also equivalent to a statistic originally proposed by Hoeffding [3]. The characteristic function of the null asymptotic distribution of $B_n$ is (cf. [1] and [3]) \begin{equation*}\tag{7}E e^{izB} = \prod^\infty_{j,k=1} \big(1 - \frac{2iz}{\pi^4j^2k^2}\big)^{-\frac{1}{2}}.\end{equation*} Asymptotic power properties are given in [1]. As in Rothman [9] the difficulty in modifying these tests in the toroidal case is that there is no natural origin for the distribution functions on a circle. Moreover, different arbitrary starting points give the test statistics $A_n$ and $B_n$ different values. In this paper we propose that the statistic $C_n$ be used for our problem, with the surface of the torus replacing the plane. \begin{equation*}\tag{8}C_n = n \int\int Z_n^2(x, y) dF_n(x, y)\end{equation*} where $Z_n(x, y) = T_n(x, y) + \int\int T_n(x, y) dF_n(x, y) - \int T_n(x, y) dF_{n1}(x) - \int T_n(x, y) dF_{n2}(y).$ We note that $C_n$ may be rewritten in the following form which we shall refer to in Section 3: \begin{equation*}\tag{9}C_n = 1/n^2 \sum^n_{j=1} \lbrack\sum^n_{a=1} \{T_n(X_j, Y_j) + T_n(X_a, Y_a) - T_n(X_a, Y_j) - T_n(X_j, Y_a)\}\rbrack^2.\end{equation*} We shall also have occasion to use the random variable, $D_n = n \int\int Z_n^{\ast^2}(x, y) dF_1(x)dF_2(y),$ where $Z_n^\ast(x, y) = T_n(x, y) + \int\int T_n(x, y)dF_1(x)dF_2(y) - \int T_n(x, y)dF_1(x) - \int T_n(x, y) dF_2(y).$ An outline of the paper follows: In Section 2 it is shown that when $H_0$ is true, $C_n - D_n \rightarrow 0$ in probability. The invariance of $C_n$ under changes of origin is proved in Section 3. Finally the asymptotic distribution of $C_n$ under the null hypothesis is obtained in Section 4.

Consider $p(\geqq 2)$ treatments in an experiment involving paired comparisons. The $(i, j)$th pair yields $n_{ij}$ observations $X_{ijm}$ with distribution functions (df's) $F_{ij}(x) = F(x - \beta_i + \beta_j), (1 \leqq m \leqq n_{ij}; 1 \leqq i < j \leqq p), \mathbf{\beta} = (\beta_1, \cdots, \beta_p)'$ being the vector involving treatment effects. We assume that $F$ is symmetric about zero, i.e. $F(x) + F(-x) = 1$, for all real $x$. This is, for example, the situation, when one considers a replicated balanced incomplete block design with each block of size two under the usual assumption of additivity in an analysis of variance model. The null hypothesis to be tested is as follows: (1.1) $H_0': F_{ij}(x) = F(x)$, for all real $x$, and $1 \leqq i < j \leqq p,$ against all alternatives, which is equivalent to (1.2) $H_0: \mathbf{\beta} = \mathbf{0},$ against $\beta \neq \mathbf{0, 0}$ being a $p$-component column vector. A class of rank-order tests for the above problem was considered in Mehra and Puri (1967) and Puri and Sen (1969b), and asymptotic distributions of test statistics were obtained both under the null hypothesis, and under a sequence of alternatives converging to the null hypothesis at a suitable rate. However, any asymptotic optimality properties of the test procedures were not considered in either of the two papers. For subsequent notational convenience, we first pool the $n = \sum\sum_{i\leqq i<j\leqq p} n_{ij}$ observations, and label the $m$th observation for the $(i, j)$th pair by $\alpha = \sum^p_{\lambda=k+1} \sum^{i-1}_{k=0} n_{k\lambda} + \sum^{j-1}_{\lambda=i} n_{i\lambda} + m(1 \leqq m \leqq n_{ij})$, where we define $n_{0\lambda} = 0, n_{ii} = 0 (1 \leqq \lambda \leqq p, 1 \leqq i < j \leqq p, 1 \leqq \alpha \leqq n)$. Also, set $c_{in\alpha} = 1, -1$ or 0 according as the $\alpha$th observation is from a block where the treatment $i$ is paired with some treatment, the index of which is $> i, <i$, or the treatment $i$ is not involved in the block at all $(1 \leqq \alpha \leqq n; 1 \leqq i \leqq p)$. Then one can write \begin{equation*}\tag{1.3}P(X_{n\alpha} \leqq x) = F_{n\alpha}(x) \text{(say)} = F(x - \mathbf{\beta'c}_{n\alpha}),\end{equation*} $\mathbf{c}_{n\alpha} = (c_{1n\alpha}, \cdots, c_{pn\alpha})', 1 \leqq \alpha \leqq n.$ We may remark at this point that the paired comparison problem can be regarded as a particular case of the more general regression problem considered in Puri and Sen (1969a). The object of the present note is to show that, while tests considered in Puri and Sen (1969a), are, under certain conditions asymptotically optimal for the regression problem (and, hence, for the paired comparison case if the same conditions are satisfied), tests considered in Mehra and Puri (1967) or Puri and Sen (1969b) are, in general, not so. A counter-example is given in Section 3 to illustrate this fact.

Conditions are established under which maximum likelihood estimators are consistent and asymptotically normal in the case where the observations are independent but not identically distributed. The key concept employed is uniform integrability; and the required convergence theorems which involve uniform integrability, and are of independent interest, appear in the appendix. A motivational example involving estimation under variable censoring is presented. This example invokes the full generality of the theorems with regard to lack of i.i.d. and lack of densities $\operatorname{wrt}$ Lebesgue or counting measure.

A discrete time, finite Markov chain with fixed initial state and stationary transition behavior is considered. Using Whittle's formula a large deviation result (similar to Hoeffding's result for one multinomial distribution) is obtained for the transition count matrix of a path of the chain of arbitrary length. This result is then used in the asymptotic comparison of a given sequence of tests about the transition probability matrix with a suitably constructed sequence of likelihood ratio tests. It is assumed that the sizes of these tests decrease to zero at a certain rate as the length of the observed path increases. The comparison is carried out at fixed alternatives in terms of the behavior of the ratio of type-II-error probabilities.

Transformations of a class of Gaussian processes to the Brownian motion are obtained by reproducing kernel Hilbert space methods. These transformations are such that the value of the transformed process at any point of time is given in terms of the sample path of the original process up to that time. In certain situations the boundary-crossing behaviors of the original process and the transformed process are related.

Let $\{X(t), -\infty < t < \infty\}$ be a stationary Gaussian process with covariance function satisfying: (1) $r(t) = 1 - C|t|^\alpha + o(|t|^\alpha)$ as $t \rightarrow 0: C > 0, 0 < \alpha \leqq 2$; and (2) $r(t) = O(t^{-\gamma})$ as $t \rightarrow \infty: \gamma > 0$. Then for all positive increasing functions $\phi(t)$ on $\lbrack a, \infty), P\lbrack X(t) > \phi(t)$ infinitely often $\rbrack = 0$ or 1 as $\int^\infty_a \phi(t)^{2/\alpha-1} \exp\{-\phi^2(t)/2\} dt < \infty$ or $= \infty$. This result generalizes the paper of Watanabe [Trans. Amer. Math. Soc. 148 233-248] by replacing his condition that $r(t) = o(1/t)$ as $t \rightarrow \infty$ by condition (2). Our result is extended also to the nonstationary process treated by Watanabe. Our proof treats the problem as a crossing problem using a recent result of Pickands [Trans. Amer. Math. Soc. 145 51-73] and a modification of the Borel lemmas.

Let $F(x)$ be an infinitely divisible distribution function with a Levy-Khintchine function $G(u)$ and let $p$ be any positive number. It is shown that $F(x)$ has an absolute moment of the $p$th order if and only if $G(u)$ has an absolute moment of the $p$th order, and $F(x)$ has an exponential moment of the $p$th order if and only if $G(u)$ has an exponential moment of the $p$th order. This result generalizes a theorem of J. M. Shapiro. Other related results are also obtained.

A necessary and sufficient condition is obtained for an $L$ function to have a continuous derivative of the $k$th order. Other results concerning the continuity properties of infinitely divisible distribution functions are also obtained.

For $n = 1,2, \cdots$ let $z^{(n)}_t, t \geqq 0$, be an age-dependent branching process starting from $n$ ancestors. Suppose it has the reproduction generating function $f_n, f_n'(1) = 1 + \alpha/n + o(n^{-1}), f_n''(1) = 2\beta_n \rightarrow 2\beta, f_n'''(1-) \leqq$ some constant, and the life-length distribution $L$ with $L(0) = 0$ and $\lambda = \int^\infty_0 tL(dt) < \infty$. Then, it is shown that the finite dimensional distributions of $n^{-1}z^{(n)}_{nt}$ converge, as $n \rightarrow \infty$, to the corresponding laws of the diffusion $t \rightarrow x_t$ with drift $(\alpha/\lambda)x$ and infinitesimal variance $(2\beta/\lambda)x$.

This article studies efficient data transmission, or "data compression", from the standpoint of the theory of epsilon entropy. The notion of the entropy of a "data source" is defined. This quantity gives a precise measure of the amount of channel capacity necessary to describe a data source to within a given fidelity, epsilon, with probability one, when each separate "experiment" must be transmitted without storage from experiment to experiment. We also define the absolute epsilon entropy of a source, which is the amount of capacity needed when storage of experiments is allowed before transmission. The absolute epsilon entropy is shown to be equal to Shannon's rate distortion function evaluated for zero distortion, when suitable identifications are made. The main result is that the absolute epsilon entropy and the epsilon entropy have ratio close to one if either is large. Thus, very little can be saved by storing the results of independent experiments before transmission.

Inequalities are obtained for the modes of $L$ functions whose Levy spectral functions have support on the positive axis. An application of these inequalities is made to certain stochastic processes.

Two results of Billingsley [Convergence of Probability Measures (1968), Wiley, New York] on weak convergence of empirical distribution functions for sequences of $\phi$-mixing random variables are shown to hold under weaker conditions.

Let $S_n = \sum^{k_n}_{k=1} X_{nk}$ and $X$ be random variables with distribution functions $F_n(x)$ and $F(x)$. No assumptions are made that the $(X_{nk})$ have finite means or variances. Also, no independence conditions are assumed. A bound is found for $M_n = \sup_{-\infty<x<\infty}|F_n(x) - F(x)|.$ This bound involves various truncated moments and conditional probabilities and expectations. A typical quantity involved is $\sum^{k_n}_{k=1} E|E (X_{n^k}|\sum^{k-1}_{j=1} X_{nj}) - E(X_{n^k})|$. Using this bound, particular conditions are found so that $S_n$ converges in distribution to $X$.

We construct in this paper a continuous, nowhere constant, square integrable martingale such that $P\{M(\frac{1}{2})^k = 0\} \geqq \frac{7}{8}$ for $k \geqq 3$. This construction is used to show that in general, $\lim_{t\rightarrow 0}\int^t_0\Phi(s)dM(s, \omega)/M(t, \omega) \neq \Phi(0)$ where $\Phi(s)$ is nonrandom and right continuous, $M(t, \omega)$ is a continuous, nowhere constant, square integrable, martingale, and the limit is a limit in probability.

Let $X_n$ be a sequence of random variables adapted to an increasing sequence of $\sigma$-fields. In this note, convergence properties of $EX_t$ are studied as $t\rightarrow\infty$ through the directed set of stopping variables. The analogue of the inequality in Fatou's Lemma turns out to be an equation, which strengthens Fatou's Lemma. These problems arise naturally in the theory of gambling.

Let $X(g)$ be a homogeneous random field on a discrete locally compact Abelian group $G$. Let $H(X)$ be the linear completion of $\{X(g): g \in G\}$ in $L_2$ space. The following result is obtained: there exists a fundamental random field $Y(g)$ on $G$ with values in $H(X)$ such that $X(g)$ is obtained as a moving average of $Y(g)$ if, and only if, $X(g)$ has a spectral density which is positive almost everywhere with respect to the Haar measure on the dual group of $G$.

The rate of first passage of the integrated Wiener process to $x > 0$ is determined in terms of the "$\frac{1}{2}$-winding time" distribution of H. P. McKean, Jr. The probability that the integrated Wiener process is currently at its maximum is approximated.