This paper is an expository survey of the mathematical aspects of statistical inference as it applies to finite Markov chains, the problem being to draw inferences about the transition probabilities from one long, unbroken observation $\{x_1, x_2, \cdots, x_n\}$ on the chain. The topics covered include Whittle's formula, chi-square and maximum-likelihood methods, estimation of parameters, and multiple Markov chains. At the end of the paper it is briefly indicated how these methods can be applied to a process with an arbitrary state space or a continuous time parameter. Section 2 contains a simple proof of Whittle's formula; Section 3 provides an elementary and self-contained development of the limit theory required for the application of chi-square methods to finite chains. In the remainder of the paper, the results are accompanied by references to the literature, rather than by complete proofs. As is usual in a review paper, the emphasis reflects the author's interests. Other general accounts of statistical inference on Markov processes will be found in Grenander [53], Bartlett [9] and [10], Fortet [35], and in my monograph [18]. I would like to thank Paul Meier for a number of very helpful discussions on the topics treated in this paper, particularly those of Section 3.

The use of a statistic of the algebraic form of Pearson's chi-squared as a measure of goodness of fit for frequencies from a fully specified $m$th order stationary Markov chain was first discussed and contrasted with the appropriate likelihood ratio criterion by Bartlett [2]. Since the distribution of the former statistic is not that of a tabular $\chi^2$-variate, it and allied statistics, are sometimes described as "psi-squared" statistics. Patankar [14] derived the approximate asymptotic distribution (as the total number of transitions $\rightarrow \infty$) of \begin{equation*}\tag{1}\psi^2_1 = \sum_i \lbrack(n_i - m_i)^2/m_i\rbrack,\end{equation*} where the $n_i$ are the marginal frequencies (1-tuples) in a large sequence from a simple stationary Markov chain and the $m_i$ are their expected values in a new sequence of the same length. The proof is based on the fact that for a large sequence of observations the marginal frequencies are asymptotically multivariate normal and then (1) is distributed as a linear function of independent $\chi^2$-variates. Since the latter can be approximated by a single Type III variate ([5], [15]) the approximate asymptotic distribution of (1) is completely specified by its first two moments. Let $n_\mathfrak{u}$ be the frequency of the $t$-tuple $\mathfrak{u} = (u_1, u_2, \cdots, u_t)$ in a sequence of length $n + t - 1$ from an $m$th order stationary Markov chain; and let $m_\mathfrak{u}$ be its expected value in a new sequence of the same length. To test whether the chain has a specified transition probability matrix, in analogy with (1) one may construct the statistic \begin{equation*}\tag{2}\psi^2_t = \sum_\mathfrak{u}\lbrack(n_\mathfrak{u} - m_\mathfrak{u})^2/m_\mathfrak{u}\rbrack\end{equation*} and test the goodness of fit for $n_\mathfrak{u}$. In (2) the summation extends over those values of $\mathfrak{u}$ for which $m_\mathfrak{u}$ does not vanish. Using methods different from those used here, Good [9] gave the asymptotic distribution of $\psi^2_t$ for the special case of a random sequence of digits, and showed that for an equiprobable random sequence (Markovity of order $-1$) having a prime number of categories, $\psi^2_t$ is asymptotically a linear combination of independent $\chi^2$-variates. This was generalized to the case of an arbitrary number of categories and to an arbitrary random sequence (Markovity of order 0) by Billingsley [4]. Good [11] conjectured that a similar result might be true for Markovity of any order. Following Good [10], Goodman [12] has shown that this conjecture is not true, and has proceeded to study a modification that is true. For further work in this direction and additional references see [13]. Since it is clear that the distribution of (2) does not have a simple form, we might assume that it follows the Type III form approximately. This approximation is suggested by the fact that $(n_\mathfrak{u})$ is asymptotically normal and hence the quadratic form (2) in $(n_\mathfrak{u})$ is distributed asymptotically as a linear function of $\chi^2$-variates with one degree of freedom [5]. Since (2) is nonnegative, the coefficients of the corresponding linear function of $\chi^2$-variates are also nonnegative. In the case when $m = 1$ or 0, the exact values of these coefficients are also known [4], [9]. The problem of approximating the distribution of a linear function of $\chi^2$-variates has been discussed by Welch [15] and Box [5]. They observe that this Type III approximation is fairly good over a wide range of values of degrees of freedom of the different $\chi^2$ and their coefficients, especially when these coefficients are positive. The advantage of this approximation is that it enables us to test the goodness of fit by referring to standard $\chi^2$-tables. In Section 2 of this paper, we derive this approximate distribution of (2) by obtaining its first two moments for any $m$ and $t \geqq m$. Let $X_1, X_2, \cdots, X_{n + t - 1}$ be a series of observations from a stationary linear Markov sequence (autoregressive) of first order; \begin{equation*}\tag{3}X_i = \rho X_{i - 1} + Y_i\quad(i = 2, 3, \cdots, n + t - 1),\end{equation*} where $|\rho| < 1$, and the $Y_i$ are independent identically distributed continuous random variables with zero mean and range $(-\infty, +\infty)$. (Even though not in universal use, the term "Markov sequence" here refers to a Markov chain with continuous state space. We follow Bartlett [3] in using it.) Let these $n + t - 1$ observations be grouped into $k$ class intervals and let $n_\mathfrak{u}$ be the frequency of the $t$-tuple $(X_{u_1}, X_{u_2}, \cdots, X_{u_t})$ in this sequence, where $X_{u_1}, X_{u_2}, \cdots, X_{u_t}$ are $t(\geqq 1)$ consecutive observations belonging to the $u_1$th, $u_2$th, $\cdots, u_t$th class intervals respectively. For these frequencies, we derive the approximate distribution of the psi-squared test defined by (2), under some mild restrictions on the distribution of $Y$ and for small class intervals, assuming $\rho$ to be known. For the case $t = 1$, and the distribution of $Y$ normal, Patankar [14] has obtained its distribution. We observe that, for $t = 1$ and $Y$ arbitrarily distributed, the same distribution is obtained. In Section 4, the distribution of the $\psi^2_t$ test of goodness of fit for frequencies of $t$-tuples $(t \geqq 2)$ in a series of observations, grouped into a finite number of class-intervals, from the stationary linear Markov sequence (autoregressive) of second order, \begin{equation*}\tag{4}X_i = aX_{i - 1} + bX_{i -2} + Y_i,\end{equation*} is derived, under similar restrictions on the distribution of $Y$. From this, the distribution of $\psi^2_t$ for stationary linear Markov sequences (autoregressive) of arbitrary order is deduced. The distribution of (2) may also be used to calculate the power of the usual $\chi^2$-test of goodness of fit for independent observations, when the alternative is serial dependence.

In a regular Markov chain with one absorbing state, for sequences starting from a given state and continuing until the absorbing state is reached, the distribution and moment formulae of the total number of transitions is derived in Section 2, and also its probability generating function (p.g.f.). The joint p.g.f. of the transition frequencies is given in Section 3, from which the p.g.f. of one or more transition frequencies is deduced. In Section 4 some moment formulae associated with these transition frequencies are derived. Section 5 is concerned with inference for such Markov chains, when there are a large number of sequences starting from the same given state.

We shall be concerned with experimental data given in the form of frequencies in cells determined by a multiway cross-classification, with predefined categories along each way of classification. Roy and Bhapkar [10] have posed hypotheses, which might be considered generalizations appropriate to this set up of the usual hypotheses in classical "normal" univariate "fixed effects" analysis of variance, "normal" multivariate "fixed effects" analysis of variance and analysis of various kinds of "normal" independence. Large sample tests for such hypotheses are offered here. The large sample tests suggested are based on the $\chi^2$-test of Karl Pearson [8]. The general probability model is that of a product of several multinomial distributions. According as the marginal frequencies along any dimension are held fixed or left free, that dimension is said to be associated with a "factor" or a "response" (or variable). The probability model is \begin{equation*}\tag{1}\prod_j \frac{n_{oj}!}{n_{ij}!} \prod p^{n_{ij}}_{ij},\end{equation*} where $\sum_i p_{ij} \equiv p_{oj} = 1$ and $\sum_i n_{ij} \equiv n_{oj}$ is held fixed. Thus $i$ refers to categories of the response while $j$ refers to categories of the factor. $n_{oj}$ denotes the preassigned sample-size for the $j$th factor-category, out of which $n_{ij}$ happen to lie in the $i$th response-category. It should be noticed that $i$ may be a multiple subscript, say $i_1, i_2, \cdots, i_k; j$ also may be a multiple subscript, say $j_1, j_2 \cdots, j_l$. We then speak of a $k$-response (or $k$-variate) and $l$-factor problem According as a set of real numbers is or is not associated with the categories along any way of classification (factor or response), that way of classification will be said to be structured or unstructured. It is well-known (for example, Neyman [6]) that if a hypothesis $H_o$ is given in the form of certain constraints on the $p_{ij}$'s, then a large sample test statistic of $H_o$ under (1) for the model is a $\chi^2$ statistic given by $\sum_{ij} (n_{ij} - n_{oj}\hat p_{ij})^2/(n_{oj}\hat p_{ij}),$ or a $\chi^2_1$ statistic given by $\sum_{ij} (n_{ij} - n_{oj}\hat p_{ij})^2/n_{ij}$, where the $\hat p_{ij}$'s form any set of BAN estimates [6]. In the particular case when the constraints are linear in $p$'s, the method of minimum $\chi^2_1$ permits a reduction of the problem to the solution of a system of linear equations and hence is more convenient. Reiersol [9] considers binomial experiments and makes use of results of Neyman [6] to determine tests for hypotheses appropriate to factorial experiments. Mitra [5] not only generalizes Reiersol's theorems to multinomial experiments, but also avoids his restriction that the parameter-sets in the different linear forms occurring in the hypothesis be nonoverlapping. We shall prove theorems to cover the cases that cannot be treated by these theorems. In Section 2, the $\chi^2_1$ statistic based on the minimum $\chi^2_1$ estimates is obtained to test linear hypotheses. It is further shown that, when $H_o$ specifies linear functions of the $p$'s as known linear functions of some unknown parameters, the $\chi^2_1$ statistic, based on the minimum $\chi^2_1$ estimates, is exactly the same as the minimum sum of squares of residuals obtained by a certain general least squares technique to estimate the unknown parameters. This is then applied to derive test criteria appropriate to various hypotheses proposed in [3] and [10].

Tables of critical values defining an unbiased test are given for testing the null hypothesis $\sigma^2 = \sigma^2_0$ against the two-sided alternative hypothesis $\sigma^2 \neq \sigma^2_0$ where $\sigma^2$ is the variance of a normal population. Use of the tabulated values leads to the logarithmically shortest confidence limits for $\sigma^k, k > 0$. The critical values have been found to five significant figures for $\alpha = .01, .05, .10$ where $\alpha$ is the size of the critical region and for $\nu = 1(1)20, 24, 30, 40, 60, 120$ where $\nu$ equals the degrees of freedom of the chi-square distribution. A least squares equation is given which may be used to find the critical values when $\nu \geqq 10$ for $\alpha = .01, .05, .10$. Since submitting a revision of the present paper for publication, the article by Tate and Klett [6] appeared necessitating a second revision. An explanation of the overlap of the present paper with [6] is included in Section 5. In addition, a brief discussion of [2], which was called to the writer's attention by the editor, has been added in Section 5.

We are given independent random samples $X_1, \cdots, X_m$ and $Y_1, \cdots, Y_n$ from populations with unknown cumulative distribution functions (cdf's) $F_X$ and $F_Y$, respectively. It is desired to test $H_o:F_X = F_Y$ against $H_1 : F_X = G_\theta,\quad F_Y = G_\phi,\quad\theta, \phi \varepsilon R,$ where $G_\theta$ is a specified family of cdf's (one for each $\theta$), $R$ is an interval on the real line, $\theta$ and $\phi$ are specified and very close to some specified value $\phi_o$, and $\theta \neq \phi$. A theorem of Hoeffding is used to show that the locally most powerful rank test (L.M.P.R.T.) of $H_o$ against $H_1$ is based on a linear rank statistic $T_N = (m)^{-1} \sum^N_{i = 1} a_{Ni}Z_{Ni},$ where $Z_{Ni} = 1$ when the $i$th smallest of $N = m + n$ observations is an $X$, and $Z_{Ni} = 0$, otherwise, and the $a_{Ni}$ are given numbers. In a recent paper, Chernoff and Savage established the asymptotic normality of the test statistic $T_N$, subject to some weak restrictions. The concept of asymptotic relative efficiency (A.R.E.) was introduced by Pitman to compare sequences of tests. It was pointed out by Chernoff and Savage that the asymptotic efficiency of a sequence of tests can be established by means of a likelihood ratio test. Using this method, in conjunction with the theorem of Chernoff and Savage on asymptotic normality, it is shown that the L.M.P.R.T. of $H_o$ against $H_1$ is asymptotically efficient. Several applications to Cauchy, exponential, and normal populations are given.

This paper extends and unifies some previous formulations and theories of estimation for one-parameter problems. The basic criterion used is admissibility of a point estimator, defined with reference to its full distribution rather than special loss functions such as squared error. Theoretical methods of characterizing admissible estimators are given, and practical computational methods for their use are illustrated. Point, confidence limit, and confidence interval estimation are included in a single theoretical formulation, and incorporated into estimators of an "omnibus" form called "confidence curves." The usefulness of the latter for some applications as well as theoretical purposes is illustrated. Fisher's maximum likelihood principle of estimation is generalized, given exact (non-asymptotic) justification, and unified with the theory of tests and confidence regions of Neyman and Pearson. Relations between exact and asymptotic results are discussed. Further developments, including multiparameter and nuisance parameter problems, problems of choice among admissible estimators, formal and informal criteria for optimality, and related problems in the foundations of statistical inference, will be presented subsequently.

An $s$ stage $k$ name snowball sampling procedure is defined as follows: A random sample of individuals is drawn from a given finite population. (The kind of random sample will be discussed later in this section.) Each individual in the sample is asked to name $k$ different individuals in the population, where $k$ is a specified integer; for example, each individual may be asked to name his "$k$ best friends," or the "$k$ individuals with whom he most frequently associates," or the "$k$ individuals whose opinions he most frequently seeks," etc. (For the sake of simplicity, we assume throughout that an individual cannot include himself in his list of $k$ individuals.) The individuals who were not in the random sample but were named by individuals in it form the first stage. Each of the individuals in the first stage is then asked to name $k$ different individuals. (We assume that the question asked of the individuals in the random sample and of those in each stage is the same and that $k$ is the same.) The individuals who were not in the random sample nor in the first stage but were named by individuals who were in the first stage form the second stage. Each of the individuals in the second stage is then asked to name $k$ different individuals. The individuals who were not in the random sample nor in the first or second stages but were named by individuals who were in the second stage form the third stage. Each of the individuals in the third stage is then asked to name $k$ different individuals. This procedure is continued until each of the individuals in the $s$th stage has been asked to name $k$ different individuals. The data obtained using an $s$ stage $k$ name snowball sampling procedure can be utilized to make statistical inferences about various aspects of the relationships present in the population. The relationships present, in the hypothetical situation where each individual in the population is asked to name $k$ different individuals, can be described by a matrix with rows and columns corresponding to the members of the population, rows for the individuals naming and columns for the individuals named, where the entry $\theta_{ij}$ in the $i$th row and $j$th column is 1 if the $i$th individual in the population includes the $j$th individual among the $k$ individuals he would name, and it is 0 otherwise. While the matrix of the $\theta$'s cannot be known in general unless every individual in the population is interviewed (i.e., asked to name $k$ different individuals), it will be possible to make statistical inferences about various aspects of this matrix from the data obtained using an $s$ stage $k$ name snowball sampling procedure. For example, when $s = k = 1$, the number, $M_{11}$, of mutual relationships present in the population (i.e., the number of values $i$ with $\theta_{ij} = \theta_{ji} = 1$ for some value of $j > i$) can be estimated. The methods of statistical inference applied to the data obtained from an $s$ stage $k$ name snowball sample will of course depend on the kind of random sample drawn as the initial step. In most of the present paper, we shall suppose that a random sample (i.e., the "zero stage" in snowball sample) is drawn so that the probability, $p$, that a given individual in the population will be in the sample is independent of whether a different given individual has appeared. This kind of sampling has been called binomial sampling; the specified value of $p$ (assumed known) has been called the sampling fraction [4]. This sampling scheme might also be described by saying that a given individual is included in the sample just when a coin, which has a probability $p$ of "heads," comes up "heads," where the tosses of the coin from individual to individual are independent. (To each individual there corresponds an independent Bernoulli trial determining whether he will or will not be included in the sample.) This sampling scheme differs in some respects from the more usual models where the sample size is fixed in advance or where the ratio of the sample size to the population size (i.e., the sample size-population size ratio) is fixed. For binomial sampling, this ratio is a random variable whose expected value is $p$. (The variance of this ratio approaches zero as the population becomes infinite.) In some situations (where, for example, the variance of this ratio is near zero), mathematical results obtained for binomial sampling are sometimes quite similar to results obtained using some of the more usual sampling models (see [4], [7]; compare the variance formulas in [3] and [5]); in such cases it will often not make much difference, from a practical point of view, which sampling model is utilized. (In Section 6 of the present paper some results for snowball sampling based on an initial sample of the more usual kind are obtained and compared with results presented in the earlier sections of this paper obtained for snowball sampling based on an initial binomial sample.) For snowball sampling based on an initial binomial sample, and with $s = k = 1$, so that each individual asked names just one other individual and there is just one stage beyond the initial sample, Section 2 of this paper discusses unbiased estimation of $M_{11}$, the number of pairs of individuals in the population who would name each other. One of the unbiased estimators considered (among a certain specified class of estimators) has uniformly smallest variance when the population characteristics are unknown; this one is based on a sufficient statistic for a simplified summary of the data and is the only unbiased estimator of $M_{11}$ based on that sufficient statistic (when the population characteristics are unknown). This estimator (when $s = k = 1$) has a smaller variance than a comparable minimum variance unbiased estimator computed from a larger random sample when $s = 0$ and $k = 1$ (i.e., where only the individuals in the random sample are interviewed) even where the expected number of individuals in the larger random sample $(s = 0, k = 1)$ is equal to the maximum expected number of individuals studied when $s = k = 1$ (i.e., the sum of the expected number of individuals in the initial sample and the maximum expected number of individuals in the first stage). In fact, the variance of the estimator when $s = 0$ and $k = 1$ is at least twice as large as the variance of the comparable estimator when $s = k = 1$ even where the expected number of individuals studied when $s = 0$ and $k = 1$ is as large as the maximum expected number of individuals studied when $s = k = 1$. Thus, for estimating $M_{11}$, the sampling scheme with $s = k = 1$ is preferable to the sampling scheme with $s = 0$ and $k = 1$. Furthermore, we observe that when $s = k = 1$ the unbiased estimator based on the simplified summary of the data having minimum variance when the population characteristics are unknown can be improved upon in cases where certain population characteristics are known, or where additional data not included in the simplified summary are available. Several improved estimators are derived and discussed. Some of the results for the special case of $s = k = 1$ are generalized in Sections 3 and 4 to deal with cases where $s$ and $k$ are any specified positive integers. In Section 5, results are presented about $s$ stage $k$ name snowball sampling procedures, where each individual asked to name $k$ different individuals chooses $k$ individuals at random from the population. (Except in Section 5, the numbers $\theta_{ij}$, which form the matrix referred to earlier, are assumed to be fixed (i.e., to be population parameters); in Section 5, they are random variables. A variable response error is not considered except in so far as Section 5 deals with an extreme case of this.) For social science literature that discusses problems related to snowball sampling, see [2], [8], and the articles they cite. This literature indicates, among other things, the importance of studying "social structure and...the relations among individuals" [2].

The bivariate normal distribution, with its numerous applications, is of considerable importance and has been studied fairly extensively. Among the first statisticians to investigate the distribution were Sheppard [12] and Karl Pearson [9], the latter from the point of view of his celebrated "tetrachoric functions", which were used as the basis for computing tables of the distribution. Pearson's tables have been extended by the University of California Statistical Laboratory [16] and, more fully, by the National Bureau of Standards [5]. In more recent years, the distribution has been studied among others by Nicholson [6], Polya [10], Cadwell [1] and Owen [7], [8]. Owen has also provided useful tables from which the bivariate normal integral may be evaluated. These tables have been published in [7] and in extended form, together with auxiliary tables, in [8]. (The reader is referred to [8] and [5] for further references and for some interesting applications.) An essential part of the procedures used by Nicholson and Owen is to reduce the integral, which is a function of three parameters, the coordinates $(x_0, y_0)$ of the vertex of the infinite rectangle over which integration is to be extended and the correlation coefficient $\rho$, to functions of only two parameters. The series based on tetrachoric functions for the bivariate normal integral suffers from the disadvantage that it converges rather slowly except when $|\rho|$ is small. The need for an expression which shall be suitable for all values of $\rho$, but more especially for high $|\rho|$, has long been felt (see e.g., David [3]). Formula (3.16), taken in conjunction with (2.7), as well as formula (3.21), is designed to meet this need. These are two-parameter formulae and have the further advantage of being especially useful for high values of $x_0$ and/or $y_0$. Next, the formulae are used to provide equivalent rapidly convergent Stieltjes type continued fractions, known as $S$-fractions (equations (4.6) and (4.19)). These two sets of formulae constitute the basic results of this paper. They are, in fact, analogues of the corresponding known formulae for the univariate normal integral.

Let $\{X_t, t \geq 1\}$ be a stochastic process which satisfies the following set of assumptions: ASSUMPTION 1: For every $t, X_t$ satisfies \begin{equation*}\tag{1}X_t = \alpha_1X_{t - 1} + \alpha_2X_{t - 2} + \cdots + \alpha_kX_{t - k} + u_t,\end{equation*} where $\alpha_1, \cdots, \alpha_k$ are $k$ finite real numbers (unknown parameters) and $u_t, t$ positive, are independent, identically distributed random variables with mean zero and a finite positive variance $\sigma^2$. ASSUMPTION 2: The distribution of $u_t$ is continuous. (Actually $\mathrm{Pr}\{u_t = 0\} = 0$ suffices.) ASSUMPTION 3: The roots $m_1, \cdots, m_k$ of the characteristic equation \begin{equation*}\tag{2}m^k - \alpha_1m^{k - 1} - \alpha_2m^{k - 2} - \cdots - \alpha_k = 0,\end{equation*} of (1), are distinct. ASSUMPTION 4: There is a unique root $\rho$ of (2) such that $|\rho| > 1$, and $|\rho| > \max_{j = 2, \cdots, k} |m_j|$. Here $\rho$ is identified with $m_1$ for convenience. Since complex roots enter in pairs, it follows from this assumption that $\rho$ is real. Note that there can be $m_j, j > 1$, such that $|m_j| > 1$. ASSUMPTION 5: For $t$ non-positive, $u_t = 0$. If Assumption 4 holds, the process $\{X_t, t \geqq 1\}$ is said to be (strongly) explosive, and the corresponding difference equation (1) is called an explosive (linear homogeneous) stochastic difference equation; this is the subject of the present paper. Under the assumptions above, it follows (cf., C. Jordan [5], p. 564, Mann and Wald [8], p. 178, and also the footnote on p. 22 of [10]) that $X_t = \sum^t_{r = 1}\sum^k_{q = 1} \lambda_qm^{t - r}_qu_r,$ $t$ positive, and that $\lambda_q$ satisfy the relations \begin{equation*}\tag{3}\delta_{1t} = \sum^k_{q = 1} \lambda_qm^{t - 1}_q,\quad t = 1, 0, -1, \cdots, - (k - 2),\end{equation*} where $\delta_{1t} = 1$ if $t = 1$ and 0 otherwise. (Note that $\sum^k_{q = 1}\lambda_q = 1$.) For convenience, define the random variables \begin{equation*}\tag{4}X_{i,t} = \sum^t_{r = 1} m^{t - r}_iu_r,\quad i = 1, 2, \cdots, k, (m_1 = \rho),\end{equation*} so that $X_{i,t} = 0$ for $t$ non-positive. Thus one may write $X_t$ as follows: \begin{equation*}\tag{5}X_t = \lambda_1X_{1,t} + \lambda_2X_{2,t} + \cdots + \lambda_kX_{k,t}.\end{equation*} The first part of this paper is devoted to finding a consistent estimator of $\rho$ and its limit distribution. Consequently, in Section 3 some lemmas will be proved for use in the consistency proof (Theorem I). Similarly, in Section 5, some lemmas leading to the proof of the limit distribution of the estimator (Theorem II) will be given. In the second part, the consistency of the Least Squares (L.S.) or Maximum Likelihood (M.L.) estimators of the "structural parameters" $\alpha_i$ of (1) will be considered (Theorem III). The procedure becomes much more involved because the direct application of the usual limit theorems is not possible, since the process under consideration is explosive. It is noteworthy that Lemmas 9, 10, 14-16, and Theorem I are rather general, in that they hold under the only global Assumptions 1-5 above, and the further requirement $|m_j| < 1, j = 2, \cdots, k$, so essential for the rest of the analysis of this paper, is unnecessary for them. The corresponding problem, in the case $|\rho| < 1$, has been completely solved by Mann and Wald [8]. If $k = 1$ in (1), the results of this paper reduce to those obtained by Rubin [13], White [14], and T. W. Anderson [1]. The vector case has also been treated by Anderson in [1], but a comparison of the results in this case with those of the present paper shows that they do not imply each other except in the first order. In the latter case, however, both reduce to Rubin's [13] result. The available results on stochastic difference equations are summarized in a table at the end of the paper. Some of the details and computations omitted in this paper may be found in [10]. In the following section, some known lemmas related to stochastic convergence are collected and stated in a convenient form, as they will be constantly referred to in both parts of the paper. (For proofs, see [2], [3], [4], [6] and [9].)

This paper considers the probabilities of first emptiness of two dams in parallel, both subject to a steady release at constant unit rate, and fed by a discrete additive input process such that unit inputs are always directed to the dam with lesser content. The problem is equivalent to that of the single dam fed by two ordered inputs, and a recurrence relation for the probabilities of first emptiness in this process is obtained. Equations for the generating functions of the probabilities are derived, and a formal solution to these is given. A more convenient method of evaluating probabilities of first emptiness is found by reducing the process to an associated occupancy problem; it is shown how the probabilities of first emptiness for Poisson inputs are then obtained by a rapid computational procedure. The paper concludes with a general formulation of the problem when the times of arrival for two ordered non-negative inputs of random size form a Poisson process.

The class of mixtures of a one-parameter additively-closed family of distributions is proved identifiable. A condition for a class of scale parameter mixtures to be identifiable is indicated and applications to Type III and uniform distributions are made.

Extending the results of Kiefer and Wolfowitz [10], [11], methods are obtained for characterizing and computing optimum regression designs in various settings, and examples are given where $D$-optimum designs are computed. In Section 1 we introduce the main definitions and notation which will be used in the paper, and discuss briefly the roles of invariance, randomization, number of points at which observations are taken, and nonlinearity of the model, in our results. In Section 2 we prove the main theoretical results. We are concerned with the estimation of $s$ out of the $k$ parameters, extending an approach developed in [10] and [11] in the case $s = k$. There is no direct way of ascertaining whether or not a given design $\xi^{\ast}$ is $D$-optimum for (minimizes the generalized variance of the best linear estimators of) the $s$ chosen parameters, and T+eorems 1 and 2 provide algorithms for determining whether or not a given $\xi^\ast$ is $D$-optimum. If all $k$ parameters are estimable under $\xi^\ast$, we can use (2.7) to decide whether $\xi^\ast$ is $D$-optimum, while if not all $k$ parameters are estimable we must use the somewhat more complicated condition (2.17) (of which part (a) or (b) is necessary for optimality, while (a), (c), or (d) is sufficient). An addition to Theorem 2 near the end of Section 3 provides assistance in using (2.17) (b). Theorem 3 of Section 2 characterizes the set of information matrices of the $D$-optimum designs. In Section 3 we give a geometric interpretation of the results of Section 2, and compare the present approach with that of [10]. In the case $s = k$, the present approach reduces to that of Section 5 of [10] and of [11]. When $1 < s < k$, we obtain an algorithm which differs from that of Section 4 of [10] and which appears to be computationally easier to use. When $s = 1$, the results of the present paper are shown to reduce to those of Section 2 of [10]; in particular, we obtain the game-theoretic results without using the game-theoretic machinery of [10]. In Section 4 we determine $D$-optimum designs for the problems of quadratic regression on a $q$-cube and polynomial regression on a real interval with $1 < s < k$. Part II of the paper is devoted entirely to the determination of $D$-optimum designs for various problems in the setting of simplex designs considered by Scheffe [12]. Various unsolved problems are mentioned throughout the paper. Further examples will be published elsewhere.

Consider experiments of the following type. Observation is made of a univariate random variable $X$ whose absolutely continuous distribution function $F (x) \mid \theta)$ and probability density function $p(x \mid \theta)$ are functions of a real unknown parameter $\theta$. Different experiments of this type with random variables $X_1, X_2, \cdots$ will be denoted $\epsilon_1, \epsilon_2, \cdots$ In the following definitions, $\Theta$ represents a subset of $\theta$-values. (a) Following Blackwell [1], $\epsilon_1$ is sufficient for $\epsilon_2$ with respect to $\Theta$ or $\epsilon_1 \succ \epsilon_2(\Theta)$ when there exists a stochastic transformation of $X_1$ (given by a set of distribution functions $\{G(z | x_1) | - \infty < x_1 < \infty\})$ to a random variable $Z$ such that, for each $\theta \epsilon \Theta, Z(X_1)$ and $X_2$ have identical distributions. (b) Following Lindley [3], $\epsilon_1$ is not less Shannon informative than $\epsilon_2$ with respect to $\Theta$ or $\epsilon_1S \geqq \epsilon_2(\Theta)$ when $\mathscr{I}\lbrack\epsilon_1, F(\theta)\rbrack \geqq |mathscr{I}\lbrack\epsilon_2, F(\theta)\rbrack$ for all "prior" distribution functions $F(\theta)$ giving probability one to $\Theta,$ where $\mathscr{I}\lbrack\epsilon_i, F(\theta)\rbrack$ is the mean Shannon information given by $\epsilon_i$ about $\theta$ when $\theta$ has the prior distribution function $F(\theta).$ (c) When the Fisher imnformations $I_i(\theta) = \int^\infty_{-\infty} p(x_i | \theta) \bigg\lbrack\frac{\partial}{\partial\theta} \log p(x_i | \theta) \bigg\rbrack^2 dx_i, \quad i = 1, 2,$ are definable for $\theta \varepsilon |Theta, \epsilon_1$ will be said to be not less Fisher informative than $E_2$ with respect to $\Theta,$ or $\epsilon_1F \geqq \epsilon_2(\Theta),$ when $I_1(\theta) \geqq I_2(\theta)$ for $\theta \varepsilon \Theta.$ Lindley [3] has shown that $\epsilon_1 \succ \epsilon_2(\Theta) \longrightarrow \epsilon_1S \geqq \epsilon_2(\Theta).$ In Theorem 1, we show that under certain conditions $\epsilon_1S \geqq \epsilon_2(\Theta) \longrightarrow \epsilon_1F \geqq \epsilon_2(\Theta).$ If this implication always held, comparison by $F \geqq$ would be more widely applicable than comparison by $S \geqq$ (and a fortiori by $succ$). However the conditions of Theorem 1 suggest that cases exist where $\epsilon_1S \geqq |epsilon_2(\Theta)$ but where $I_1(\theta)$ and $I_2(\theta)$ are not even defined for $\theta \varepsilon \Theta.$ When $\theta$ is a location parameter, $p(x \mid \theta) = f\lbrack x - \theta\rbrack,$ say. For fixed $f\lbrack \cdot \rbrack$ consider the class of experiments $\{\epsilon(c) \mid c > 0\},$ where $\epsilon(c)$ is the experiment determined by the probability density function cf$\lbrack c(x - 0)\rbrack.$ The conditional distribution of $\epsilon(c_1)$ is a contraction of that of $\epsilon(c_2)$ when $c_1 > c_2.$ (Example: $\epsilon(c)$ consisting of $c^2$ observations from the normal distribution $N(\theta, 1)$ and $x$ their mean). In the theorems of Sections 3, 4 and 5, conditions for $\epsilon(c_1) \succ \epsilon(c_2), \epsilon(c_1)S \geqq \epsilon(c_1)F \geqq \epsilon(c_2)$ when $c_1 > C_2$ are given. Unless otherwise indicated integrals will be taken over $R^1.$

An exponential bound is obtained for the law of large numbers for $S_n = \sum^n_{k=1} f(X_k)$ where $\{X_k: k = 1, 2, \cdots \}$ is a discrete parameter Markov process satisfying Doeblin's condition and $f$ is a bounded, real-valued, measurable function.