Tests of simple and composite hypothesis for multinomial distributions are considered. It is assumed that the size $\alpha_N$ of a test tends to 0 as the sample size $N$ increases. The main concern of this paper is to substantiate the following proposition: If a given test of size $\alpha_N$ is "sufficiently different" from a likelihood ratio test then there is a likelihood ratio test of size $\leqq\alpha_N$ which is considerably more powerful than the given test at "most" points in the set of alternatives when $N$ is large enough, provided that $\alpha_N \rightarrow 0$ at a suitable rate. In particular, it is shown that chi-square tests of simple and of some composite hypotheses are inferior, in the sense described, to the corresponding likelihood ratio tests. Certain Bayes tests are shown to share the above-mentioned property of a likelihood ratio test.

During the past five or six years there has been much work done on the problem of representing excessive functions as the potentials of additive functionals. This is a natural generalization to Markov processes of the classical theorem of Riesz dealing with the representation of non-negative superharmonic functions as the potentials of measures. These results (due mainly to Meyer [9], Sur [12], and Volkonski [13]) are now in a fairly definitive state, and the purpose of this paper is to give a cohesive account of them. In addition to their intrinsic importance these representation theorems have been very useful in several recent applications, for example, in the theory of time changes [2] and the theory of local times [4]. Although the general theory of Markov processes is extremely rich, it is necessary to set up a rather large amount of notation and machinery before coming to grips with the problems of interest. Consequently Section 1 contains a compact summary of the definitions and basic theorems of what is now called the theory of Hunt processes, that is, Markov processes satisfying Hypothesis (A) of Hunt's fundamental memoir [8]. The author feels that such a summary is worthwhile in itself, since it can serve as a source of definitions and notations for many current research papers. The reader familiar with this material should begin with Section 2 and refer back to Section 1 only as needed. The proofs of most of the results quoted in Section 1 can be found in [1], [6], [11], or, of course, [8]. Beginning with Section 2 the exposition becomes more leisurely. We have tried not only to state definitions and theorems, but also to give some insight into them and indicate some of the more important (in our opinion) open problems. The proofs of all results quoted in Sections 2-5 (with the exception of 2.6.1 (ii)) can be found in [7], as well as in the original papers cited in connection with each theorem.

First an integral representation of a continuous linear functional dominated by a support function in integral form is given (Theorem 1). From this the theorem of Blackwell-Stein-Sherman-Cartier [2], [20], [4], is deduced as well as a result on capacities alternating of order 2 in the sense of Choquet [5], which includes Satz 4.3 of [23] and a result of Kellerer [10], [12], under somewhat stronger assumptions. Next (Theorem 7), the existence of probability distributions with given marginals is studied under typically weaker assumptions, than those which are required by the use of Theorem 1. As applications we derive necessary and sufficient conditions for a sequence of probability measures to be the sequence of distributions of a martingale (Theorem 8), an upper semi-martingale (Theorem 9) or of partial sums of independent random variables (Theorem 10). Moreover an alternative definition of Levy-Prokhorov's distance between probability measures in a complete separable metric space is obtained (corollary of Theorem 11). Section 6 can be read independently of the former sections.

Many statistical problems contain an infinite parameter space or are analyzed as if they contained one. In a Bayesian analysis of such a problem, there is often, for one or more of several reasons, an attraction in employing an infinite measure on parameter space in the role of `prior distribution' of the parameter. The employment of such a quasi prior distribution consists in its formal substitution as the prior density in Bayes's theorem to produce a quasi posterior distribution. (We will qualify the "posterior distribution," obtained in this way, as quasi, even if it is a probability distribution with integral one, as will be assumed for the rest of this paper). The attractions referred to are that the quasi prior distribution (i) may be thought to represent "ignorance" about the parameter; (ii) may give (quasi) posterior distributions satisfying some "natural" invariance requirement; (iii) may itself satisfy some "natural" invariance requirement (the Jeffreys invariants); (iv) may give (quasi) posterior distributions on the basis of which statistical statements may be constructed which closely resemble those of classical statistics. [A separate argument for the quasi prior distribution is that, for an infinite parameter space, the class of Bayes decision functions may be complete only if the class includes those derived from quasi prior distributions (e.g. Sacks (1963)); but in this paper we will go no further than consideration of posterior distributions.] In the foundations of Bayesian statistics, associated with the names of Ramsey, de Finetti and Savage (but not Jeffreys), quasi prior distributions do not appear. When finally arrived at, subjective prior distributions are finite measures. Moreover they are, for any given person, uniquely determined, so that there should be no question of choice. Hence, matters such as the representation of ignorance, invariance and the degree of resemblance to classical statistics are not relevant. But it is possible to accept this standpoint and then to argue that the consequences of using quasi prior distributions are worth investigating if only as convenient approximations in some sense. As Welch [(1958), p. 778] reveals, such an attitude must have been implicitly adopted by those nineteenth century followers of Bayes and Laplace who ascribed a probability content to the interval between probable error limits of some astronomical or geodetic observation. (With a normal distribution of known variance taken for the observations, the implicit quasi prior distribution was, of course, uniform on the real line.) One sense of the "approximation" is straightforward. Imagine fixed data. A quasi prior distribution may be a satisfactory approximation to the actual prior distribution for this set of data if it locally resembles or simulates the actual distribution over some important compact set of parameter points determined by the data; the fact that the quasi prior distribution is not integrable on the complement of this compact set may not be important. For example, if, in sampling from a normal distribution, the quasi prior resembles the actual prior in the region where the likelihood function is not close to zero, then the approximation may be satisfactory. Thus, for given data, actual prior distribution and criterion of satisfactory approximation, the question of deciding whether a certain quasi prior distribution is employable is theoretically straightforward, the answer being based on a direct comparison of the actual posterior and quasi posterior distributions. Furthermore, even without knowledge of the data, a calculation of the prior probability (evaluated by the actual prior distribution) of obtaining data for which the quasi prior distribution is employable will provide the necessary prospective analysis. However, when no actual prior distribution is given, two courses of justification of a given quasi prior distribution are available. One, suggested by a referee, would consist of the demonstration that, for each member of a wide class of proper (and possibly actual) prior distributions, the corresponding posterior distributions are (with high probability) satisfactorily close to the quasi posterior distribution. The other, which formally avoids any decision as to when two distributions are satisfactorily close to each other, is asymptotic and would ask the question "Does there exist a sequence of proper prior distributions such that, as we proceed down the sequence, the posterior distributions converge in some sense to the quasi posterior distribution?" Jeffreys [(1957), p. 68] and Wallace [(1959), p. 873] have adopted the latter course. Wallace shows without difficulty that, roughly speaking, given a quasi prior distribution, there exists a sequence of proper prior densities whose corresponding posterior densities tend to the quasi posterior density for each fixed set of data. Reintroducing the concept of satisfactory approximation, the existence of this type of convergence, which may be called Jeffreys-Wallace convergence, assures us that, for all reasonable criteria of approximation, given the data there will be a member of the constructed sequence of prior distributions whose corresponding posterior distribution will be satisfactorily approximated by the quasi posterior distribution. Since this particular prior distribution could be the actual prior distribution of some experimenter, a certain justification of the quasi posterior distribution is thereby provided. This justification can be made separately for each set of data thereby yielding an apparently prospective justification of the quasi prior distribution itself (that is, a justification of its use for all sets of data). However, it is clear that the justification is essentially retrospective, since the prior selected may depend on the data or, in other words, the convergence (of posterior distributions to quasi posterior distribution) may not be uniform with respect to different data. In this paper, we will, like Jeffreys and Wallace, adopt the asymptotic justification course (fearing that the more statistically relevant alternative is exceedingly complex) but our justification will be genuinely prospective. To obtain this prospective asymptotic justification of a given quasi prior distribution, we may, perhaps, impose the condition that the convergence be uniform. However, even where it is possible, such a requirement is stronger than is statistically necessary. All we need is convergence in probability defined as follows. (The precise definition, applied to a special context, is given later.) There is convergence in probability to the quasi posterior distributions corresponding to a given quasi prior distribution if there exists a sequence of proper prior distributions such that, at each parameter point, the corresponding sequence of posterior densities converges in probability to the quasi posterior density. The latter use of "converges in probability" is the customary one with the Bayesian slant that the sequence of probability distributions with respect to which it is defined are the marginal distributions of data corresponding to the sequence of prior distributions. From this definition it is clear that, for all reasonable criteria of approximation, for each $\epsilon > 0$ there will be a member of the postulated sequence of prior distributions such that the prior probability of obtaining data for which the quasi posterior distribution is a satisfactory approximation to the posterior distribution corresponding to the member will exceed $1 - \epsilon$. Since this member could be the actual prior distribution of some experimenter, a genuinely prospective justification of the given quasi prior distribution is thereby provided. Wallace's theorem shows that there is Jeffreys-Wallace convergence to all quasi prior distributions, so that all quasi prior distributions are asymptotically justified in the Jeffreys-Wallace sense. However, no equivalent theorem is available for convergence in probability, which requires demonstration for each quasi prior distribution considered. The above distinction has been drawn previously by Stone (1963), (1964) for data from the normal distribution-univariate and multivariate. The present work provides a generalisation of the case. Two analytical restrictions are made. The first, essential for the results obtained, is that the experiment generating the data should have the property of invariance under a group of transformations. (Following Fraser (1961), the supposed invariance will generally be conditional on an ancillary statistic). Many statistical problems have this group invariance structure. The second restriction (which may well be inessential) is that, given any quasi prior distribution, only sequences of prior distributions obtained by truncations of the quasi distribution to compact parameter sets will be considered. In Section 2, the group invariant structure of the experiment is outlined. In Section 3, we follow Hartigan (1964) by introducing relatively invariant prior distributions. In Section 4, convergence in probability in this context is defined and in Theorems 4.1 and 4.2 it is shown that right Haar measure (as quasi prior distribution) is, under certain conditions, sufficient and necessary (among relatively invariant prior distributions) for convergence in probability. In Section 5, general statistical applications are considered.

A problem considered in this paper is to obtain confidence intervals for all characteristic roots of a population covariance matrix $\mathbf{\Sigma}$ in the form $\lbrack ch_m(\mathbf{S})/u, ch_M(\mathbf{S})/l\rbrack$, where $ch_m(\mathbf{S})$ and $ch_M(\mathbf{S})$ are the minimum and maximum characteristic roots, respectively, of a sample covariance matrix $\mathbf{S}$ from a multivariate normal population and $u$ and $l$ are constants. Intervals of this form having probability at least $1 - \epsilon$ can be obtained by basing $u$ and $l$ on certain $\chi^2$-distributions. Among all intervals in a certain class such intervals are shortest. Another problem treated is to obtain confidence intervals for all characteristic roots $ch(\mathbf{\Sigma}_1\mathbf{\Sigma}_2^{-1})$ in the form $\lbrack ch_m(\mathbf{S}_1\mathbf{S}_2^{-1})/U, ch_M(\mathbf{S}_1\mathbf{S}_2^{-1})/L\rbrack$, where $\mathbf{\Sigma}_1$ and $\mathbf{\Sigma}_2$ and $\mathbf{S}_1$ and $\mathbf{S}_2$ are population and sample covariance matrices of two multivariate normal populations, respectively, and $U$ and $L$ are constants, determined from $F$-distributions to give confidence at least $1 - \epsilon$. Such choices of the constants yield shortest intervals within a certain class. Comparison is made with other methods of finding such intervals. Various uses of the intervals are suggested, such as simultaneous intervals for variances and correlation coefficients. Some other confidence intervals for related problems are considered.

Suppose we have an input $X(t)$ made up of an unknown waveform $\theta(t)$ of known length, which is repeated randomly, and is imbedded in Gaussian noise with a known covariance function. The rate of recurrence of the waveform is a known small constant. In addition, the signal-to-noise ratio of the input $X(t)$ is quite low. We wish to estimate the waveform $\theta(t)$ and its autocorrelation $\psi(\tau) = \int \theta(t + \tau)\theta(t) dt$. Restricting ourselves to discrete-time observations on $X(t)$, we shall derive an optimal estimator of the discrete version of $\psi(\tau)$. This estimator is a weighted average of the sample autocorrelation and the square of a linear estimator of the time average (the zero-frequency or DC value) of the waveform. For the estimation of $\theta$, the problem is more complicated. The optimality concept (asymptotic efficiency) used in this work is based upon large-sample theory and the Cramer-Rao Inequality (Chernoff [2] and Cramer [4]). The problem stated above was motivated by a problem of electronic surveillance of an enemy communication system based upon pulse position modulation, PPM. To illustrate this system suppose station $A$ is sending a message, which we wish to intercept and decode, to station $B$ by PPM over a certain FM bandwidth. $A$ continues to repeat a fixed pulse-type waveform $\theta(t) = \sum^n_{i = 1} \theta_iH\lbrack t - (i - 1)T\rbrack$ where $H(t) = 1\quad\text{if} \quad 0 \leqq t < T,\\ = 0\quad\text{otherwise}$. The vector $\theta' = (\theta_1, \cdots, \theta_n)$, the parameter $n$, and $T$--the pulse width--are known to both $A$ and $B$. Notice that $nT$ is the time duration (length) of $\theta(t)$. Since $\theta, n,$ and $T$ are known to the receiver $B$, one may ask what the coding scheme is for the information that $A$ is sending. The answer is that the length of time between successive recurrences of the waveform, is the variable which contains the information. In many applications the average length of time between successive occurrences of $\theta(t)$ is around $10^2$ times $nT$. While the intervals between repetitions are fundamental in the transmission of information, someone who does not yet know the waveform may regard the recurrences to be purely random in time. This modulation technique has the effect of spreading the power in the FM bandwidth over a wider swath of the frequency scale. This makes surveillance more difficult because it requires that we somehow determine the actual bandwidth being used. Moreover, the spreading of the power makes jamming of the channel difficult. To further complicate matters, $A$ transmits the pulses with low power so that $B$ picks up an input $X(t)$ with low signal-to-noise ratio. Since $B$ knows $\theta(t)$, he uses matched filtering to detect the times of occurrence of the $\theta's$. If the noise is assumed to be additive Gaussian noise with known covariance, then matched filtering is optimal in a decision theoretic sense (Wainstein and Zubakov [8]). But suppose that we are listening in on this channel without knowing $\theta$ and we wish to find out what $A$ is saying. First we must determine the frequency band of the channel which $A$ is using. Then we must detect the times of occurrence of the $\theta's$, although we do not know $\theta$. However, let us assume that we have already determined $n$ and $T$, although it will turn out that $n$ is not a vital parameter in the estimator developed in this paper. Jakowatz, Shuey, and White [6] present a special discrete-time (sampled data) system, called the Adaptive Filter, which estimates an unknown waveform which is repeating in additive noise. The system uses a complicated stochastic iterative procedure. The Filter obtains a crude estimate of the waveform from the initial input and uses the discrete cross-correlation between this estimate and the input to detect the times of occurrence of the waveform. When it decides that a waveform is present in the input, it refines the estimate by averaging it with the section of input where the waveform is thought to be present. Provided the autocorrelation of the waveform--$\psi(\tau)$--has $\psi(0)$, its maximum, a good deal larger than the relative maxima of $\psi$, and provided the noise is well behaved, then this iterative procedure results in an asymptotically stable estimate for the waveform. This stable estimate is then used as the matching element in matched filter detection of the waveform. Thus we could call the Adaptive Filter an adaptive matched filter. A partial analysis of this system is given by Hinich [5]. The estimate of the discrete autocorrelation of the waveform is helpful to the analysis of systems which are based upon discrete-time cross-correlation such as the Adaptive Filter, since the autocorrelation is a basic parameter in the distributions of the random variables (correlations) which arise in the operation of these systems. Suppose we can obtain an expression for $\theta$ in terms of $\psi, \theta = f(\psi)$. Once we have obtained an asymptotically efficient estimator of $\psi$, call it $\hat\psi$, then $\hat\theta = f(\hat\psi)$ would be an asymptotically efficient estimator of $\theta$. Unfortunately, there is a multiplicity of $\theta's$ which have $\psi$ as their autocorrelation. In Section 6 we will present a method for obtaining the correct $\theta$ (the one which appears in the input) from $\psi$ by using the observations on $X(t)$. Unfortunately this method does not seem to be sufficiently practical. To conclude, let us outline the rest of this paper. In Section 2 we give a formal statement and description of the problem posed above. In Section 3 we state the Cramer-Rao theorem and derive the information matrices relevant to the estimation of $\theta$ and $\psi$, as well as their inverses. In Section 4 we present the optimal estimator of $\psi$. We also show that while the normalized sample correlation is an unbiased estimator of $\psi$, it is not efficient. In Section 5 we discuss three examples. These are the general case white noise, the case of white noise when the discrete waveform has only two components $\theta_1, \theta_2$, and the case of Gaussian Markov noise $(EN (t + \tau)N(t) = \rho^{|\tau|}, 0 < \rho < 1)$. In Section 6 we discuss the problem of estimating $\theta$ after the autocorrelation $\psi$ has been estimated.

In this paper we consider the mean number of crossings of an arbitrary curve, in a given time $T$, by a non-stationary normal process. A formula (2) is obtained for this and sufficient conditions for its validity given. These conditions concern the behaviour of the covariance function of the normal process. Incidental results include sufficient conditions for continuity of a normal (non-stationary) process $x(t)$ and also for $x(t)$ to satisfy a Holder condition.

In a previous paper ([2], Theorem 3.1) this author has given some sufficient conditions for a stationary process to be a function of a finite Markov chain. Suppose $\mathscr{F}$ denotes the class of functions which satisfy these conditions. In this paper we give a characterisation of $\mathscr{F}$. Using this characterisation $\mathscr{F}$ is shown to be wider than the class of regular functions of finite Markov chains.

As is well known ([7], [12], [13] and [14]) Markov processes may be studied via an appropriate potential theory, which for symmetric random walks is discrete Newtonian potential theory (see [8], [19]). Here we will be concerned with the problem of deciding when a set of lattice points in $s \geqq 3$ dimensions is recurrent, that is, when it is visited infinitely often a.s. by the symmetric random walk. In [14] a necessary and sufficient condition was given to decide this problem. Intuitively this test determines whether or not $\infty$ is a regular point of a lattice set (see [11]). However the test tells one very little about the recurrence properties of an arbitrarily given lattice set. Here we desire more precise information about lattice sets. Namely can one impose regularity conditions on lattice sets such that there exists a weighting $\mu(a)$, such that the divergence of $\sum_{a\epsilon A} \mu(a)$ is necessary and sufficient for the recurrence of the lattice set $A$? As a step in this direction we give a necessary and sufficient condition for recurrence of a set for a general Markov chain in terms of the existence of a non-negative solution to a Wiener-Hopf type equation on the set in question. As a byproduct of this equation for finite lattice sets, we obtain interesting bounds for the probabilistic capacity of finite sets as well as explicit expressions for the probability of leaving a finite set forever. As an application of our criterion for recurrence in $s = 3$ dimensions, regularity conditions are given on lattice sets so that $\mu(a) = |a|^{-1}$, where $|a|$ is the Euclidean distance from the lattice point $a$ to the origin, is the appropriate weighting so that $\sum_{a\varepsilon A} \mu(a) = \infty \iff A$ is recurrent. Further it is shown that the regularity conditions cannot be removed, as we exhibit a set for which the above series is divergent but the set is not recurrent. Further the above regularity conditions are invariant under arbitrary possibly different rotations of each lattice point about the origin and hence recurrent lattice sets satisfying these regularity conditions remain recurrent under arbitrary rotation. Finally a necessary condition and a sufficient condition are given for subsets of the axis in 3 dimensions.

The purpose of this paper is to establish the following theorem and several corollaries to it. THEOREM 1. Let $\{\xi_k : k = 0, \pm 1, \cdots\}$ be an independent sequence of real valued random variables with $E\xi_k = 0$ and moment generating functions $f_k(t) = Ee^{t\xi_k}$ such that: (1) for every $\beta > 0$ there exists $T_\beta > 0$ such that $f_k(t)$ exists and $|1 - f_k(t)| \leqq \beta|t|$ for $|t| \leqq T_\beta$ uniformly in $k$. Let $\{a_{n,k} : k = 0, \pm 1, \cdots; n = 1, 2, \cdots\}$ be real numbers such that: (2) $\sum^\infty_{k = -\infty} |a_{n,k}| < A < \infty$ for $n = 1, 2, \cdots$ (3) $f(n) = \sup_k |a_{n,k}| \rightarrow 0$ as $n \rightarrow \infty$. Then $S_n = \lim_{a\rightarrow-\infty,b\rightarrow\infty} \sum^b_{k = a} a_{n,k}\xi_k$ is defined as an almost sure limit for all $n$, and for every $\epsilon > 0$ there exists a positive $\rho_\epsilon < 1$ (depending on $A$ but not on the particular $a_{n,k}'s)$ such that \begin{equation*}\tag{(4)}P\lbrack|S_n| \geqq \epsilon\rbrack \leqq 2\rho_\epsilon^{1/f(n)}.\end{equation*} This theorem is applied in Section 3 to establish exponential convergence rates for the strong law of large numbers for subsequences of linear processes of non-identically distributed random variables. In Section 4, the application of the theorem to the summability theory of sequences of independent random variables is discussed. Section 2 is devoted to proving the theorem.

Certain statistical problems in life testing and reliability lead to the consideration of minimizing and maximizing certain integrals of the form $\int^\infty_0 \phi(x, \bar F(x)) dx,\quad\bar F(x) = 1 - F(x)$, under the assumption that the distribution function $F$ has increasing hazard rate and that one or two moments are given. The minimizing (maximizing) distributions for the special cases considered are members of the extremal class considered by A. Marshall and the author [2]. Bounds on the expected values of the minimum and maximum order statistics as well as the operating characteristics of replacement policies based on age are obtained under these assumptions.

$\{X_n\}$ is a sequence of random variables defined on some probability space, $\mathscr{P} = \{P_\theta, \theta\epsilon \Theta\}$ is the family of distributions of $\{X_n\}$ and $\mathscr{A}_n$ is the subfield generated by $(X_j, j \leqq n)$. It is assumed that $\Theta$ is a real interval, $X_n \rightarrow \theta$ a.e. $P_\theta$ and that, for each $n, X_n$ is sufficient for $\mathscr{P}$ on $\mathscr{A}_n$ while $\mathscr{P}$ is a homogeneous monotone likelihood ratio family on $\mathscr{A}_n$. Let $p_{\theta n}$ denote the density of $P_\theta$ with respect to some $\sigma$-finite measure on $\mathscr{A}_n$ and consider the sequence $\{R_n\}$, where $R_n = p\theta_2n/p\theta_1n$ and $\theta_1 < \theta_2$. Conditions are given for the occurrence of the following limiting behavior of $R_n : (1)$ there exists a $\theta_0, \theta_1 < \theta_0 < \theta_2$, such that $R_n \rightarrow 0$ or $\infty$ a.e. $P_\theta$ according as $\theta <$ or $> \theta_0$ and (2) $P_{\theta_0} (b < R_n < a) \rightarrow 0$ for $0 < b < a < \infty$. This limiting behavior of $R_n$ guarantees the termination with probability one of sequential probability ratio tests based on $\{X_n\}$. Let $q_{\theta n}$ denote the density of $P_\theta$ on the Borel field generated by $X_n$. In order to describe the results of this paper we introduce Condition $A_1$ which states, essentially, that $n^{-1} ln \lbrack q_{\theta n} (x)/K(n)\rbrack \rightarrow h(\theta, x)$ for some functions $K$ and $h$ satisfying very mild conditions. Condition $A_2$ requires $h(\theta, x)$ to possess, for each fixed $x$, a unique maximum at $\theta = x$. It is shown that under condition $A_1$, part (1) of the above limiting behavior of $R_n$ is equivalent to the statement that $h(\theta, x)$ satisfies Condition $A_2$ with $\theta_0$ being the solution of $h(\theta_2, x) = h (\theta_1, x)$. Moreover, under Condition $A_1$, part (2) is implied by Conditions $A_3$ and $A_4$, where Condition $A_3$ states that $q_{\theta_2^n}(\theta_0 + c/n)/q_{\theta_1^n}(\theta_0 + c/n) \rightarrow \alpha e^{c\beta}$ for some $\alpha, \beta > 0$ and all $c \neq 0$, while Condition $A_4$ requires $n^{\frac{1}{2}}(X_n - \theta_0)$ to possess under $P_{\theta_0}$ a limiting distribution which is continuous at 0. For the purpose of checking the above conditions we study the asymptotic behavior of generalized Laplace integrals. The results enable us to assert that a certain Condition B (stronger than Condition $A_1$) obtains, where $B$ states, essentially, that $q_{\theta n}(x) \sim K(n)C(\theta, x)e^{nh(\theta,x)}$ for some functions $K, C$, and $h$ satisfying mild conditions. The same techniques enable us to verify Condition $A_3$ also. The applications include the sequential $t-, F-, \chi^2-, T^2-$ and other tests.

We are given a set of $N$ responses $Y_t$ which have arisen from a nonlinear regression model \begin{equation*}\tag{(1.1)}Y_t = f(x_t, \theta) + e_t; \quad t = 1, 2, \cdots, N.\end{equation*} Here $x_t$ denotes the $t$th fixed input vector of $k$ elements giving rise to $Y_t$, whilst $\theta$ is an $m$-element unknown parameter vector with elements $\theta_i$ and the $e_t$ are a set of $N$ independent error residuals from $N(0, \sigma^2)$ with $\sigma^2$ unknown. The expectations of the $Y_t$, are therefore the functions $f(x_t, \theta)$ which will be assumed to satisfy certain regularity conditions. The problem is to estimate $\theta$ notably by least squares. In this paper we shall develop an iterative method of solution of the least squares equations which has the following properties: (a) the computational procedure is convergent for finite $N$; (b) the resulting estimators are asymptotically $100{\tt\#}$ efficient as $N \rightarrow \infty$. In Sections 2-4 we give a survey of our results leaving the mathematical proofs to Sections 5-7 whilst in Section 8 we illustrate our method with an example. Although our theoretical development is oriented towards our specific goals certain results are proved in a somewhat more general form. Some of our theory will be seen to correspond to well known theorems on stochastic limits which have to be reproved because of certain modifications which we require.

Let $\xi$ be a complex Gaussian random variable with mean $E(\xi) = \alpha$ and Hermitian positive definite complex covariance matrix $\Sigma = E(\xi - \alpha)(\xi - \alpha)^\ast$, where $(\xi - \alpha)^\ast$ is the adjoint of $(\xi - \alpha)$. Its probability density function is given by \begin{equation*}\tag{(0.1)}p(\xi \mid \alpha, \Sigma) = \pi^{-p}(\det \Sigma)^{-1} \exp \lbrack -(\xi - \alpha)^\ast\Sigma^{-1}(\xi - \alpha)\rbrack,\end{equation*} with $E(\xi - \alpha)(\xi - \alpha)' = 0$. Write \begin{equation*} \Sigma =\begin{pmatrix}\Sigma_{11} & \Sigma_{12} \\ \Sigma^\ast_{12} & \Sigma_{22}\end{pmatrix},\end{equation*} where $\Sigma_{22}$ is the $(p - 1) \times (p - 1)$ lower right-hand submatrix of $\Sigma$. Goodman (1963) found the maximum likelihood estimate of $\Sigma$ and $\rho^2 = \Sigma_{12}\Sigma^{-1}_{22}\Sigma^\ast_{12}/\Sigma_{11}$ when $\alpha = 0$ and also found the distributions of these estimates. The problems considered here are of (i) testing the hypothesis $H_{01} : \alpha = 0$ that the mean of the vector $\xi$ is 0 against the alternative $H_1 : \alpha^\ast\Sigma^{-1}\alpha > 0$ and (ii) of testing the hypothesis $H_{02} : \Sigma_{12} = 0$ that the first component of $\xi$ is independent of the others against the alternative $H_2 : \rho^2 > 0$. Since likelihood ratio test has some optimum properties and has been found satisfactory for similar problems in the real case, we find the likelihood ratio tests of these problems and show that these tests possess certain optimum properties which are counterparts of the real case. These results will be presented in Sections 3 and 4. Section 1 deals with some known results of complex matrix algebra. In Section 2, we will prove some preliminary results which are useful for complex Gaussian statistical analysis. For an application of these results the reader is referred to Goodman (1963). It may be remarked here that the likelihood ratio test is invariant under all transformations which leave the problem invariant and may be obtained from the densities of maximal invariant under the null hypothesis and the alternative.

The multivariate sampling distribution theory underlying a multivariate nornal law has a significant role in Multivariate Statistical Analysis. Several methods are available for the derivation of the usual sampling distributions, see, e.g., [2], [7], [8], [9], and [10]. This is yet another attempt in the same direction. However, the method which we present in this paper is elegant and straightforward, as the desired distributions are obtained in a rather unified way, either directly from the probability law of the sample or from the Wishart distribution. Our method is based on a generalization of Sverdrup's lemma [12] which we give below. The generalized Sverdrup's lemma is indeed implicit in many derivations in multivariate statistical literature without its explicit statement, see, e.g., Anderson's derivation of the integral representation of the noncentral Wishart distribution ([1], p. 417). Our purpose in this paper is to make an explicit statement of this implied lemma, and point out that the lemma stated here may be used as a powerful tool in multivariate distribution theory. Since our method is easily understood, we give only two applications. Several other applications follow on similar lines, see, Kabe [3], [4]. We have used fairly standard notation in this paper.

Positive definite quadratic forms in normal variates, which do not necessarily reduce to a multiple of a $\chi^2$ variate, arise quite naturally in estimation and hypothesis testing problems related to normal distributions and processes. A classical example is the problem of testing the difference between two sample means, $\bar x - \bar y$, where the $x$ observations have variance other than of $y$ observations (Welch [9]). More recent examples include: analysis of variance when errors are assumed to have unequal variance or are correlated (Box [1] [2]); regression analysis with stationary errors (Siddiqui [8]); and estimation of spectral density functions of stationary processes (Freiberger and Grenander [4]). Let $Q = \frac{1}{2}Y'MY$, where $Y = \lbrack Y_1, \cdots, Y_n\rbrack$ is a $N(0; V)$ distributed column vector, $Y'$ its transposed row vector, 0 zero vector, $V$ a positive definite covariance matrix, and $M$ a real symmetric matrix of rank $m \geqq n$. Let $a_1, \cdots, a_m$ be the non-zero characteristic roots of $A = MV$. It is well known (see, for example, Ruben [8]) that there exists a non-singular transformation from $Y$ to $X$ such that $X_1, \cdots, X_n$ are independent $N(0, 1)$ variates and $Q = \frac{1}{2} \sum^m_1 a_jX^2_j$. Without loss of generality we therefore assume that $Q$ has this canonical form. Many papers have been written on the distribution of $Q$, especially when $a_j$ are positive, and a more or less comprehensive list of these is included in the references of the two papers by Ruben [8] [9]. We shall therefore refer to only those which have direct bearing with the present paper. In this paper, we will be mainly concerned with distribution of $Q$ when $m$ is an even number, say $2k$, and $a_j$ positive. When $m$ is an odd number a slight modification is necessary and this is mentioned in Remark (2) of Section 3. We will choose our subscripts so that $0 < a_1 \leqq a_2 \cdots \leqq a_{2k}$. After some notation and preliminaries in Section 2, a well known result will be stated as Theorem 1 under which $F(x) = Pr (Q > x)$ can be evaluated as a finite linear combination of gamma df's. In other situations we require some methods of approximating to $F(x)$. In Sections 3 and 4 a simple approximation to $F(x)$ will be presented which reduces to the exact distribution when the condition of Theorem 1 is satisfied. The method is based on bounding $Q$ by $Q_1$ and $Q_2$, where $Q_1$ and $Q_2$ are quadratic forms satisfying the condition of Theorem 1. The approximation is then obtained by minimizing $d(F, \hat F)$ where $\hat F(x)$ is a linear combination of $F_i(x) = Pr (Q_i > x), i = 1, 2$, and $d(\cdot, \cdot)$ is the distance function of the metric space $L^2(0, \infty)$. In Section 5 a few numerical examples will be worked out for purposes of illustration.

This paper presents some theorems for real matrices which should be of use and interest to anyone working with quadratic forms in normally distributed variables because of the following equivalences: (i) stochastic independence of forms and orthogonality of their matrices (Craig [3] and Carpenter [1]), (ii) $\chi^2$ distribution of a form and idempotency of its matrix (Carpenter [1]), and (iii) the distribution of a form as the unique difference of two stochastically independent $\chi^2$ distributions and the tripotency of its matrix (i.e., $A = A^3$). The last equivalence is a consequence of the first two and the fact that every symmetric matrix is the unique difference of two non-negative, orthogonal symmetric matrices. The above equivalences apply specifically to forms in independently distributed variables with common variance, but this is inconsequential since if the random vector $X$ has covariance matrix $V$, then $Y = V^{-\frac{1}{2}}X$ has covariance matrix $I$ and $X'AX = Y'V^\frac{1}{2} AV^\frac{1}{2}Y$. Hence the matrix theorems of this paper can be easily adjusted to fit the more general case. The matrix theorems of this paper emphasize the use of the trace of a matrix, a concept which has not been fully exploited in such results as these. As an introductory remark, we mention that throughout this manuscript all matrices are assumed to be symmetric except for those involved in Lemma 3; and our application of that lemma will be to symmetric matrices only.

The decision procedures suggested in [1] for finite matrix games are shown to extend successfully to closed and bounded convex $S$-games, and, with some loss of desirable properties, to the general decision situation.

Let $X_1$ and $X_2$ be independent random variables (r.v.'s) and assume that $Y = X_1 + X_2$ has finite second moment. We assume that the mean and variance of $X_1$, conditional on fixed values $y$ of $Y$, satisfy the structural relations $(i) E(X_1 \mid Y = y) = \lambda_1y/\lambda\quad\text{and} (ii) V(X_1 \mid Y = y) = (\lambda_1\lambda_2/\lambda^2)u(y)$ where $\lambda_1$ and $\lambda_2$ are positive constants, $\lambda = \lambda_1 + \lambda_2$, and $u(y)$ is non-negative. Laha [2] has given a simple necessary and sufficient condition for the regression $E(X_1 \mid Y = y)$ to be linear, as we assume in (i). We use the added condition (ii) to determine explicitly the distribution functions (d.f.'s) of $X_1$ and $X_2$ (and hence of $Y$) for various choices of $u(y)$. We prove in Section 2 a theorem on which our characterizations are based and illustrate the theorem in Section 3.

In an earlier paper [3], one of us considered certain limiting distributions of response probabilities arising from the two experimenter-controlled events learning model of Bush and Mosteller [1]. There, as here, the probabilistic model took the form of a Markov process $p_0, p_1 \cdots$ satisfying the following conditions: (i) $p_0$ has an arbitrary distribution on (0, 1); (ii) if $p_n$ is given, then $p_{n+1} = a_1 + \alpha_1p_n$ with probability $\pi_1$ and $p_{n+1} = a_2 + \alpha_2p_n$ with probability $\pi_1$; (iii) $\pi_1 + \pi_2 = 1, 0 \leqq a_j \leqq 1$ and $0 \leqq \alpha_j \leqq 1 - a_j, (j = 1, 2)$. The random variable $p_n$ is called "the response probability on trial $n$." It has been shown by Karlin [2] that a limiting distribution exists as $n \rightarrow \infty$. If $p$ is the random variable of this limiting distribution, it can be shown ([1], p. 98) that the distribution of $p$ is concentrated on $\lbrack\min (\lambda_1, \lambda_2), \max (\lambda_1, \lambda_2)\rbrack$, where $\lambda_1 = a_1/(1 - \alpha_1)$ and $\lambda_2 = a_2/(1 - \alpha_2)$. In the present note, it is shown that for the case $\alpha_1 = \alpha_2 = \alpha, \pi_1 = \pi_2 = \frac{1}{2}$, the characteristic function of the distribution of the random variable \begin{equation*}\tag{1.1}x = (\lambda_1 + \lambda_2 - 2p)/(\lambda_2 - \lambda_1),\quad (\lambda_1 \neq \lambda_2),\end{equation*} when suitably standardized, tends to the characteristic function of the standardized normal distribution as $\alpha \rightarrow 1$.

The Student $t$ distribution has the property that the distribution evaluated from $-u$ to $+u$ is an increasing function of $\nu$, the degrees of freedom (this also applies to the distribution evaluated from $- \infty$ to $+ u$). (An elementary proof of this property, due to M. Ray Mickey, is based on consideration of the ratio between the Student density functions for two consecutive values of the degrees of freedom, and makes use of the inequality [5], $(1/4\nu) - (1/360\nu^3) \leqq \log a_\nu \leqq - (1/4\nu) + (2/45\nu^3)$, where $a_\nu$ is the constant in the Student $t$ density with $\nu$ degrees of freedom.) It is pointed out in this note that this monotonicity does not generalize to $k$ dimensions in a usual multivariate $t$ distribution; for $k$ sufficiently large, the direction of the monotonicity becomes reversed.