Let $y_1, y_2, \cdots$ be a sequence of random variables with known joint distribution. We are allowed to observe the $y$'s sequentially. We must terminate the observation process at some point, and if we stop at time $n$, we receive a reward which is a known function of $y_1, \cdots, y_n$. Our decision to stop at time $n$ is allowed to depend on the observations we have previously made but may not depend on the future, which is still unknown. We are interested in finding stopping rules which maximize our expected terminal reward. More formally, let $(x_n, F_n)_{1 \leqq n}$ be a stochastic sequence on a probability space $(W, F, P)$, i.e., let $(F_n)$ be an increasing sequence of sub-sigma-algebras of $F$ and for each $n \leqq 1$ let $x_n$ be a random variable (rv) measurable $F_n$. In terms of the intuitive background of the preceding paragraph, $F_n = B(y_1, \cdots, y_n)$; and although it is convenient to keep this interpretation in mind, our general results do not depend on it. A stopping rule or stopping variable (sv) is a rv $t$ with values $1, 2, \cdots, +\infty$, such that $P(t < \infty) = 1$ and for each $n \geqq 1 (t = n) \varepsilon F_n. x_t$ is (up to an equivalence) a rv, and if $v = \sup Ex_t$, where the supremum is taken over all sv's such that $Ex_t$ exists, we are interested in answering the following questions: (a) What is $v$? (b) Is there an optimal sv, i.e., one for which $Ex_t$ exists and equals $v$? (c) If there exists an optimal sv, what is it? The problem stated above is not sufficiently well formulated, as the class of sv's $t$ such that $Ex_t$ exists may be vacuous. To avoid this and other uninteresting complications we shall add the assumption that $E|x_n| < \infty, n \geqq 1$. We recall that the essential supremum (e. sup) of a family of rv's $\{q_t, t \varepsilon T\}$ is a rv $Q$ such that (1) $Q \geqq q_t$ a.s. $t \varepsilon T$, and (2) if $Q'$ is any rv such that $Q' \geqq q_t$ a.s., $t \varepsilon T$, then $Q' \geqq Q$ a.s. It is known that the essential supremum of a family of rv's always exists and can be assumed to be the supremum of some countable subfamily (e.g., [12], p. 44). Let $C_n$ be the class of all sv's $t$ such that $P(t \geqq n) = 1$ and $EX^-_t < \infty$. Let $f_n = e. \sup_{t \varepsilon C_n} E(x_t \mid F_n), v_n = \sup_{C_n} Ex_t$. It is known (Theorem 2 of [3]) that if $v < \infty$ and an optimal rule exists, then $s = \text{first} n \geqq 1$ such that $x_n = f_n$ is an optimal rule. For this and various other reasons which will become apparent, e.g., Theorem 1 below, it is desirable to have a constructive method for computing the $f_n$. The technique of backward induction and taking limits, originating with [1] and described in Theorem 2 below, achieves the desired result under certain conditions (see Theorem 2 of [4] for a general statement of these conditions). The central theorem of Section 2 provides completely general methods for computing the $f_n$. Although it seems unlikely that one would ever find it desirable to carry out these computations, there are, nevertheless, several interesting applications of the results to the theory of optimal stopping rules, and it is these applications which concern us throughout the remainder of this paper. In the course of these investigations we find it convenient to introduce the notion of an extended sv, i.e., we drop the requirement that $t$ be finite with probability one while defining $x_\infty$ to be $\lim \sup x_n$. We show that $\bar f_n = f_n$, where, relative to the class of extended sv's, $\bar f_n$ is defined analogously to $f_n$. We utilize extended sv's as a technical device within the framework of the usual theory and give examples which illustrate the inherent value of these sv's. In Section 3 we define the Markov case. We show that by paying proper attention to the Markovian structure of many stopping rule problems we are able to simplify somewhat the general theory and to give relatively simple descriptions of optimal rules when they exist. We also define randomized sv's and show that randomization does not increase $v$. We then apply this result to prove the monotonocity and continuity of $v = v(p)$ in the case where $x_n$ is the proportion of heads in $n$ independent tosses of a coin having probability $p$ of heads on each toss.

Suppose that $x_1, x_2, \cdots$ are random variables with means $\mu_1, \mu_2, \cdots$ such that \begin{equation*}\tag{1}n^{-1}(\mu_1 + \cdots + \mu_n) \rightarrow \mu\quad (0 < \mu < \infty),\end{equation*} and let $s_n = x_1 + \cdots + x_n (n \geqq 1)$. For any positive non-decreasing and eventually concave function $f$ defined on the positive real numbers and $c > 0$ let $\tau = \tau(c) = \text{first}\quad n \geqq 1\quad\text{such that}\quad s_n > cf(n)\\= \infty\quad\text{if no such}\quad n\quad\text{exists}.$ We are interested in finding conditions on $f$ and on the joint distribution of $(x_n)$ which insure that if $\lambda = \lambda(c)$ is defined by \begin{equation*}\tag{2}\mu\lambda = cf(\lambda)\end{equation*} (since we shall assume below that $f(n) = o(n), \lambda(c)$ is unique for sufficiently large $c$), then \begin{equation*}\tag{3}\lim_{c \rightarrow \infty}\lambda^{-1}E\tau = 1.\end{equation*} The elementary renewal theorem states that (3) holds when $x_1, x_2, \cdots$ are iid non-negative random variables and $f(n) \equiv 1$. Chow and Robbins [3] have obtained generalizations of this result to the case in which $(x_n)$ are dependent or non-identically distributed. The case in which $f$ is not constant has been discussed in [2], [4], and [11]. We shall assume that $x_1, x_2, \cdots$ are independent and prove the Theorem. Let $(x_n), (\mu_n), f, \tau, c, \lambda$ be as above, and suppose that for some $\alpha\varepsilon(0, 1)$ and $L$ slowly varying \begin{equation*}\tag{4}f(n) \sim n^\alpha L(n).\end{equation*} If for every $\epsilon > 0$ \begin{equation*}\tag{5}n^{-1} \sum^n_1 \int_{\{x_i - \mu_i > i\epsilon\}} (x_i - \mu_i) \rightarrow 0,\quad n \rightarrow \infty,\end{equation*} then \begin{equation*}\tag{6}\lim\sup\lambda^{-1} E \tau \leqq 1.\end{equation*} If in addition to (5) $s_n/n \rightarrow \mu$ a.s. or if for every $\epsilon > 0$ \begin{equation*}\tag{7}n^{-1} \sum^n_1 \int_{\{|x_i - \mu_i| > n\epsilon\}} |x_i - \mu_i| \rightarrow 0,\quad n \rightarrow \infty,\end{equation*} \begin{equation*}\tag{8}\sup_n E |x_n - \mu_n| \leqq K < \infty,\end{equation*} then (3) holds.

The paper gives a self-contained account of linear least squares regression when the errors have an arbitrary error covariance matrix. The finite sample size case is treated algebraically by methods which are entirely analogous to those used for the asymptotic study of the same problem by spectral analysis when the errors are generated by a covariance stationary process. The algebraic methods and results are of interest in themselves and may also be useful as an introduction to the difficult analysis involved in the asymptotic treatment.

This paper is devoted to some problems related to asymptotic power and efficiency of tests of Kolmogorov-Smirnov type. The results are derived by means of the theory of stochastic processes. The concepts of local asymptotic power and local asymptotic efficiency of tests are introduced according to [7]. Section 4 contains the evaluation of the local asymptotic power of one-sided tests of Kolmogorov-Smirnov type and of Renyi's one-sided test. The asymptotic power of these tests may be expressed by series expansion. Some theoretical results related to terms of the series are given and used for approximations and bounds of the asymptotic power. The paper is an excerpt from the author's dissertation written with the guidance of Professor Jaroslav Hajek.

Birnbaum, Esary and Marshall (1966) have shown that the class $\mathscr{F}$ of distributions with increasing failure rate averages $(IFRA)$ characterizes the concept of wear-out in the sense that $\mathscr{F}$ is the smallest class that contains the exponential distributions and is closed under the formation of coherent systems. In this note, statistical inference for models in which the distributions are unknown and $IFRA$ will be considered. Let $F$ and $G$ be defined by \begin{equation*}\tag{1.1}F(t) = H(t/\theta) \text{and} G(t) = H(t/\gamma)\end{equation*} where $H$ is an unknown $IFRA$ distribution with $H(0) = 0$. Then, for the two-sample problem where one tests the equality of the means of $F$ and $G$, it is shown that the Savage (1956) statistic maximizes the minimum power over $IFRA$ distributions asymptotically. This asymptotic minimax solution is extended to censored samples and it turns out that the Gastwirth (1965) modified version of the Savage statistic is asymptotically minimax for this case. Asymptotic uniqueness of these minimax solutions holds only in a class of rank tests. The results are extended to obtain an estimate of the ratio of the means that minimizes the maximum asymptotic variance over $IFRA$ distributions. Moreover, the results are shown to hold also for distributions with increasing failure rates $(IFR)$, extensions to the $k$-sample problem are given, and asymptotic efficiencies of the best test for exponential models are given.

Let $X_\alpha = (X_{1\alpha}, X_{2\alpha}, \cdots, X_{p\alpha}), \alpha = 1, 2, \cdots, m$, and $\alpha = m + 1, \cdots, N$, be independent random samples of sizes $m$ and $n = N - m$ from continuous $p$-variate cdf $\Psi^{(1)}(\mathbf{x})$ and $\Psi^{(2)}(\mathbf{x})$ respectively. For testing $H_0:\Psi^{(1)} = \Psi^{(2)}$ against shift alternatives $\Psi^{(1)}(\mathbf{x} - \Delta) = \Psi^{(2)}(\mathbf{x})$ various tests are available in literature of which the important ones are the following: (a) the classical Hotelling's $T^2$ test; (b) the multivariate version of the Wilcoxon test proposed by Chatterjee and Sen ([4], [5]). The test statistic $W$ is a quadratic form in the vector of coordinatewise Wilcoxon statistics. (c) The multivariate version of the normal score test proposed by Bhattacharyya [1] and for the general multisample problem by Tamura [9] and Sen and Puri [8]. The test statistic $M$ is a quadratic form in coordinatewise normal score statistics. Both $M$ and $W$ are members of the class of tests (4.7) of Tamura [9]. For consideration of asymptotic relative efficiency (ARE) let $\Psi(\mathbf{x})$ denote the common cdf under $H_0$ and $\Delta_N = \delta/N^{\frac{1}{2}}, (\delta \neq 0)$ a sequence of shift alternatives tending to $H_0$ along the direction $\delta$. Using $e_{A:B}(\delta, \Psi)$ as a general notation for the Pitman efficiency of a test $A$ relative to a test $B$, which typically depends on $\Psi$ and $\delta$, we have under suitable regularity conditions (c.f. Theorem 3, Tamura [9]) \begin{equation*}\tag{1.1}e_{M:W}(\delta, \Psi) = \Delta_M\Delta^{-1}_W,\quad e_{M:T}(\delta, \Psi) = \Delta_M\Delta^{-1}_T\end{equation*} where \begin{equation*}\tag{1.2}\Delta_M = \delta\Lambda^{-1}\delta',\quad \Delta_W = \delta\Gamma^{-1}\delta',\quad\Delta_T = \delta\Sigma^{-1}\delta'\end{equation*} and $\Lambda = (\lambda_{ij}), \Gamma = (\gamma_{ij}), \Sigma = (\rho_{ij}\sigma_i\sigma_j)$ are nonsingular $p \times p$ matrices. $\Sigma$ is the covariance matrix of $\Psi$. Denoting by $\psi_i$ and $\Psi_i$ the $i$th marginal density and cdf of $\Psi$, by $\Psi_{ij}$ the joint $(ij)$th marginal cdf and by $\phi$ and $\Phi$ the density and cdf of standard normal distribution, the typical elements of $\Lambda$ and $\Gamma$ are given by: \begin{equation*}\tag{1.3}\lambda_{ij} = \rho^\ast_{ij}\theta^{-1}_i\theta^{-1}_j \text{and} \gamma_{ij} = \rho'_{ij}\gamma_j\gamma_i\end{equation*} where \begin{equation*}\tag{1.4}\theta_i = \int^\infty_{-\infty} \lbrack\psi^2_i(x) dx/\phi\{\Phi^{-1}\lbrack\Psi_i(x)\rbrack\}\rbrack,\quad \gamma_i = \lbrack(12)^{\frac{1}{2}} \int^\infty_{-\infty}\psi^2_i(x) dx\rbrack^{-1};\end{equation*} \begin{equation*}\tag{1.5}\rho^\ast_{ij} = \int^\infty_{-\infty} \int^\infty_{-\infty} \Phi^{-1} \lbrack\Psi_i(x)\rbrack\Phi^{-1}\lbrack\Psi_j(y)\rbrack d\Psi_{ij}(x, y);\end{equation*} \begin{equation*}\tag{1.6}\rho'_{ij} = 12 \int^\infty_{-\infty} \int^\infty_{-\infty} \Psi_i(x)\Psi_j(y) d\Psi_{ij}(x, y) - 3.\end{equation*} $(\rho_{ij}), (\rho'_{ij})$ and $(\rho^\ast_{ij})$ are respectively the product moment, the grade, and the normal score correlation matrices of $\Psi$. In this paper the ARE properties of the $M$ test relative to the $W$ and $T^2$ tests are studied by investigating the bounds of the efficiencies (1.1) for various important classes of multivariate distributions. Since the ARE expressions (1.1) for two sample tests are essentially of the same structure as those for their multisample analogues, no generality is lost, as far as efficiency bounds are concerned, by restricting discussions to the two sample situation. It is shown that in the class of nonsingular multivariate normal distributions the $M$ test has efficiency 1 relative to $T^2$ and its efficiency relative to $W$ exceeds 1 irrespective of the direction $\delta$. The $M$ test behaves very well when the parent distribution has marginal densities dropping down to zero discontinuously at either tail and also in gross error models when heavy tails are present in the contaminating distribution.

This paper introduces two tests of linearity against convexity in regression. In the first, the test statistic is the number of positive signs of second differences computed from certain of the observations. In the second, a Wilcoxon statistic is computed from those differences. Possible competitors of these tests are the usual least-squares $t$-test applied to regression coefficients, Mood's median test [12], and Hill's $R$ test [6]. Certainly the first of these is to be preferred when errors are independent and normally distributed with common variance, and the alternative is quadratic regression. The sign test to be introduced here is simpler to compute than any of these other three tests, and the Wilcoxon test is also rather simple to compute. Both can be criticized in that their test statistics are calculated from certain randomly chosen observations. The tests based on second differences are compared with the $t$-test when the alternative is quadratic regression and errors are continuously and symmetrically distributed. To be precise, in the model \begin{equation*}\tag{1}Y_i = gX^2_i + bX_i + a + \epsilon_i\end{equation*} for $i = 1, \cdots, N$, we shall compare tests of $H_0:g = 0$ against $H_1:g > 0; a, b$, and $g$ are unspecified, and the $\epsilon$'s are independent, with identical distributions which are symmetric about their mean value of zero and have (unknown) variance $\sigma^2$. The criterion whereby tests are compared is Pitman efficiency, which is defined as follows. Suppose $\theta$ is an unknown real parameter of a probability distribution $H_\theta$. Suppose further that for each positive integer $N, A_N$ and $A^\ast_N$ are two size $\alpha (0 < \alpha < 1)$ tests of the null hypothesis $\theta = \theta_0$ against the alternative $\theta > \theta_0$ based on a random sample of size $N$ from $H_\theta$. Let $\beta_N(\theta)$ and $\beta^\ast_N(\theta)$ be the respective power functions, $\beta$ be a fixed number in $(\alpha, 1), \xi_N$ be a sequence of numbers for which $\xi_N \downarrow \theta$, and $M_1(\xi_N)\lbrack M_2(\xi_N)\rbrack$ be the least integer for which $\beta_{M_1}(\xi_N) \geqq \beta\lbrack\beta^\ast_{M_2}(\xi_N) \geqq \beta\rbrack$. The Pitman efficiency of $A_N$ relative to $A^\ast_N$ for the sequence of alternatives $\xi_N$ is defined to be the $\lim_{N \rightarrow \infty} M_2(\xi_N)/M_1(\xi_N)$ provided that limit exists and does not depend on $\alpha$ and $\beta$ beyond the requirement $0 < \alpha < \beta < 1$. This definition differs from some others which are commonly used (see, for example, [8]). Various technical facts concerning the Pitman efficiency of one-sample tests are discussed in an appendix, which may be of independent interest.

Universally efficient procedures for testing and estimation problems have been briefly explored by Hajek [3] and Stein [7]. In this paper we consider two populations having frequency functions $f(x)$ and $f(x - \theta)$ where the common form $f$ and the shift parameter $\theta$ are unknown. A method of estimating $\theta$ when one sample is reduced to a frequency distribution over a given set of class-intervals is suggested by the likelihood principle and the asymptotic efficiency of this estimator relative to the appropriate maximum likelihood estimator based on the complete data is found to be the ratio of the Fisher-information in a grouped observation to the Fisher-information in an ungrouped observation.

A nonparametric solution is given for the problem of selecting a subset of $k$ populations which with high probability contains the one with the largest $\alpha$-quantile, $0 < \alpha < 1$. It is assumed that each population has associated with it a continuous cdf $F_i(x) (i = 1, 2, \cdots, k)$, that samples of equal sizes are taken from each population and that observations from the same or different populations are all independent. We use the notation of the companion paper [7] without redefining all the symbols and, in particular, it is convenient to identify each population by its cdf. A procedure for the problem, based on order statistics is proposed in Section 2 and limitations on its feasibility in terms of a bound $P_1$ on the possible guarantee probability $P^\ast$ are examined in Section 3. An asymptotic expression in terms of tabled functions is also given for $P_1$. The expected size of the selected subset is examined exactly in Section 4 and asymptotically in Section 5; the latter also contains an asymptotic formula for the smallest sample size $n$ required to meet the $P^\ast$-guarantee. Asymptotic relative efficiency evaluations of the proposed procedure are carried out in Section 5. Section 6 treats the dual problem of selecting the smallest $\alpha$-quantile; Section 7 verifies a monotonicity property. Extensive tables giving $P_1$ and the integer constant defining the procedure are provided for the case $\alpha = \frac{1}{2}$.

A locally asymptotically most powerful test for a composite hypothesis $H:\xi = \xi_0$ has been developed for the case where the observable random variables $\{X_{nk}, k = 1, 2, \cdots, n\}$ are independently but not necessarily identically distributed. However, their distributions depend on $s + 1$ parameters, one being $\xi$ under test and the other being a vector $\theta = (\theta_1, \cdots, \theta_s)$ of nuisance parameters. The theory is illustrated with an example from the field of astronomy.

In practical situations, one is generally faced with multivariate problems in the form of testing the hypotheses or obtaining a set of simultaneous confidence bounds on certain parameters of interest. We shall consider here the variates under study to be normally distributed. A lot of work on the univariate and multivariate normal populations for the simultaneous confidence bounds on the location and scale parameters has been done, (see references, not necessarily exhaustive). Establishing certain inequalities for normal variates, we try to give shorter confidence bounds on variances and on a given set of linear functions of location parameters when this set is previously chosen for study. For the univariate case, Dunn [6], [8] using the Bonferroni inequality, obtained shorter confidence bounds when the number of linear functions is not too large. We may note that Nair [12], David [5], Dunn [6], [7], [8] and Siotani [22], [24] have studied the closeness of the Bonferroni inequality while deriving the percentage points of certain statistics in univariate and multivariate normal cases. In this paper, we improve the Bonferroni inequality in all the situations considered by Siotani [22], [23], [24] and Dunn [6], [7], [8], and point out various uses of these results in obtaining simultaneous confidence bounds on variances and on linear functions of means (or location parameters) with confidence greater than or equal to $1 - \alpha$ where $\alpha$ is the size of the test. We mention our main results in Section 2 for those who are interested in results and not in proofs. Since our results are extensions of Dunn [6], [8], Siotani [22], [24] and Banerjee [2], [3], their comments on the shortness of the confidence bounds apply to our cases too.

Let $\phi$ be a real valued parameter for the exponential family having densities of the form \begin{equation*}\tag{0.1}p_\phi(x) = C(\phi) \exp \lbrack\phi R(x)\rbrack\end{equation*} with respect to a $\sigma$-finite measure $\mu$ over a Euclidean sample space. Now assume that the parameter $\phi$ has a prior density $\rho(\phi)$. The posterior density of $\phi$, given $(X_1, X_2, \cdots, X_n) = (x_1, x_2, \cdots, x_n)$, is proportional to \begin{equation*}\tag{0.2}\lbrack C(\phi)e^{\phi r}\rbrack^n\rho(\phi)\quad\text{where}\quad r = \sum^n_{i = 1} R(x_i)/n.\end{equation*} The expression (0.2) is proportional to a density function and hence defines a random variable $\phi$ whose density depends on $r$. But since $r = \sum^n_{i = 1} R(x_i)/n$, the distribution of $X$ when $\phi = \phi_0$ generates a sequence $(x_1, x_2, \cdots)$ and a sequence $(R(x_1), \frac{1}{2}\lbrack R(x_1) + R(x_2)\rbrack, \cdots)$ and ultimately an infinite sequence of posterior densities of $\phi$. It is the asymptotic form of this sequence with which we shall be concerned. Neglecting for the moment the stochastic aspect of $r$, we see that if $r$ is held fixed, perhaps at the expected value of $R(x)$, the density proportional to (0.2) is exactly of the form considered in Johnson (1966, 1967). Accordingly, after suitably centering and scaling, we obtain an asymptotic expansion having the standard normal cdf as the leading term. Closely related to this approach is the work by Bernstein and von Mises. Their results are for the Bernoulli situation, and both use the usual parameter $p$ rather than the $\phi = \log \lbrack p/(1 - p)\rbrack$ which results if the density is cast into the form (0.2). von Mises actually holds $r$ fixed as he passes to the limit. Their results, which give only the limiting normal term, are reproduced in Bernstein (1934), page 406, and von Mises (1964), Chapter VIII, Section C. von Mises also gives the multinomial generalization. A more recent work following the same line of attack is given in Gnedenko (1962), Section 65. LeCam, in two basic papers (1953) and (1958), takes into account the stochastic nature of $(x_1, x_2, \cdots)$, and his Theorem 7 (1953) and Lemma 5 (1958) show that under very general conditions, the scaled posterior distribution converges to the normal distribution for almost all sequences $(x_1, x_2, \cdots)$ with respect to the infinite product measure generated by (0.1). His conditions include a more general likelihood than ours and the case where the parameter is multidimensional. See LeCam (1953), page 278, for a discussion concerning the historical background on the problem of convergence of the scaled posterior distribution. The main theorem of this paper is given in Section 1 and the following two sections give the details needed to modify the approach of Johnson (1966, 1967) so that it works in the present situation. This theorem shows that when the observations are taken from the population having density (0.1) with $\phi = \phi_0$, not only does the centered and scaled posterior distribution converge to the normal but there exists an asymptotic expansion in powers of $n^{-\frac{1}{2}}$. Section 4 gives the first two correction terms of the expansion together with examples. Throughout this work, we will use the following notational conventions. $\Phi$ and $\varphi$ are the standard normal cdf and pdf respectively. $F_n(\cdot, r)$ is the cdf of $n^{\frac{1}{2}}\theta$ where $\theta$ is defined below by Equation (1.1).

Throughout this paper we are concerned with the problem of estimating a real parameter when the loss function is such that the Bayes estimate exists, is unique, and satisfies a simple Equation, (1.5). If the estimate is unbiased (in the general sense of Lehmann [3]) we show under weak conditions that it must satisfy another Equation, (1.14). The main result of Section 1, Theorem 1.3, shows that, in general, these two equations are incompatible unless the Bayes risk is 0. This extends Theorem 11.24 of [1] which states that in estimation with quadratic loss, unbiased Bayes estimates have Bayes risk 0. Some counter-examples at the end of the section indicate the limits of this incompatibility result.

In 1949, M. Kac defined a statistic which appears useful for statistical problems arising in insurance, biology, and telephone engineering [2]. In those fields, the natural observation period is a fixed time period during which a random number of observations would be obtained. The distribution of the number of observations is given by a Poisson distribution. Distributions of this statistic can be used to determine upper and lower confidence contours for an unknown distribution, or in testing a distribution hypothesis. The purpose of this note is to extend the authors' earlier results, [1], to the two-sided Kac statistic.

A probability distribution on an $n$-dimensional Euclidean space can be specified by giving either the joint distribution of the usual rectangular coordinates or by the joint distribution of the polar coordinates. The object of this note is to obtain a characterization of normality in terms of independence of these coordinates under suitable regularity conditions. Further, if we slightly relax these conditions we get a new class of distributions which includes the normal distribution.