The analysis of variance is usually regarded as being concerned with sums of squares of numbers and independent quadratic forms of random variables. In this paper, an alternative interpretation is discussed. For certain classes of dispersion models for finite or infinite arrays of random variables, a form of generalized spectral analysis is described and its intuitive meaning explained. The analysis gives a spectral decomposition of each dispersion in the class, incorporating an analysis of the common variance, and an associated orthogonal decomposition of each of the random variables. One by-product of this approach is a clear understanding of the similarity between the spectral decomposition for second-order stationary processes and the familiar linear models with random effects.

The paper gives a contribution to the problem of finding optimal linear regression designs, when only $s$ out of $k$ regression parameters are to be estimated. Also, a treatment of design admissibility for the parameters of interest is included. Previous results of Kiefer and Wolfowitz (1959), Karlin and Studden (1966) and Atwood (1969) are generalized. In particular, a connection to Tchebycheff-type approximation of $\mathbb{R}^s$-valued functions is found, which has been known in case $s = 1$. Strengthened versions of the results are obtained for invariant designs in situations, when invariance properties of the regression setup are available. Applications are given to multiple quadratic regression and to one-dimensional polynomial regression.

$C_p, C_L$, cross-validation and generalized cross-validation are useful data-driven techniques for selecting a good estimate from a proposed class of linear estimates. The asymptotic behaviors of these procedures are studied. Some easily interpretable conditions are derived to demonstrate the asymptotic optimality. It is argued that cross-validation and generalized cross-validation can be viewed as some special ways of applying $C_L$. Applications in nearest-neighbor nonparametric regression and in model selection are discussed in detail.

In both density and regression estimation problems, the $k$-nearest neighbor estimators with $k$ varying in an appropriate range, when transformed to continuous time stochastic processes, are shown to have a common limiting structure under the usual second-order smoothness conditions as the sample size tends to $\infty$. These results lead to asymptotic linear models in which BLUE's and suitably biased linear combinations are considered.

Let us consider the class of all unimodal densities defined on some interval of length $L$ and bounded by $H$; we shall study the minimax risk over this class, when we estimate using $n$ i.i.d. observations, the loss being measured by the $\mathbb{L}^1$ distance between the estimator and the true density. We shall prove that if $S = \operatorname{Log}(HL + 1)$, upper and lower bounds for the risk are of the form $C(S/n)^{1/3}$ and the ratio between those bounds is smaller than 44 when $S/n$ is smaller than 220$^{-1}$.

Suppose we want to estimate an element $f$ of the space $\Theta$ of all decreasing densities on the interval $\lbrack a; a + L \rbrack$ satisfying $f(a^+) \leq H$ from $n$ independent observations. We prove that a suitable histogram $\hat{f}_n$ with unequal bin widths will achieve the following risk: $\sup_{f \in \Theta} \mathbb{E}_f \big\lbrack \int|\hat{f}_n(x) - f(x)|dx \big\rbrack \leq 1.89(S/n)^{1/3} + 0.20(S/n)^{2/3}$, with $S = \operatorname{Log}(HL + 1)$. If $n \geq 39S$, this is only ten times the lower bound given in Birge (1987). An adaptive procedure is suggested when $a, L, H$ are unknown. It is almost as good as the original one.

The purpose of this paper is to examine the properties of the histogram when the cells are allowed to be arbitrary. Given a random sample from an unknown probability density $f$ on $I$, we wish to construct a histogram. Any partition of $I$ can be used as cells. The optimal partition minimizes the mean integrated squared error (MISE) of the histogram from $f$. An expression is found for the infimum of MISE over all partitions. It is proved that the infimum is attained asymptotically by minimizing MISE over a class of partitions of locally equisized cells.

Elaborating on the work of Ibragimov and Has'minskii (1981) we prove a law of large deviations (LLD) for $M$-estimators, i.e., those estimators which maximize a functional, continuous in the parameter, of the observations. This LLD is applied, using the results of Petrov (1975), to the problem of parametrical nonlinear regression in the situation of discrete time, independent errors and regression functions which are continuous in the parameter. This improves a result of Prakasa Rao (1984).

A first-order autoregressive process, $Y_t = \beta Y_{t - 1} + \epsilon_t$, is said to be nearly nonstationary when $\beta$ is close to one. The limiting distribution of the least-squares estimate $b_n$ for $\beta$ is studied when $Y_t$ is nearly nonstationary. By reparameterizing $\beta$ to be $1 - \gamma/n, \gamma$ being a fixed constant, it is shown that the limiting distribution of $\tau_n = (\sum^n_{t = 1}Y^2_{t - 1})^{1/2}(b_n - \beta)$ converges to $\mathscr{L}(\gamma)$ which is a quotient of stochastic integrals of standard Brownian motion. This provides a reasonable alternative to the approximation of the distribution of $\tau_n$ proposed by Ahtola and Tiao (1984).

We show that the empirical quantile process from an ARMA$(1, q)$ process which is strongly mixing $\Delta_s$, and is either Gaussian or double exponential, converges to a Gaussian process. This result is used to derive model-free one-step-ahead prediction intervals for such processes. Simulations demonstrate where the asymptotic theory can and cannot be applied to small samples.

This paper considers the problem of sequential point and fixed-width confidence interval estimation of the location parameter when the errors form an autoregressive process with unknown scale and autoregressive parameters. The sequential point estimator considered here is based on sample mean and is shown to be asymptotically risk efficient as the cost per observation tends to zero. The sequential interval estimator is shown to be asymptotically consistent and the corresponding stopping rule is shown to be asymptotically efficient as the width of the interval tends to zero.

A class of simple adaptive allocation rules is proposed for the problem (often called the "multi-armed bandit problem") of sampling $x_1, \cdots x_N$ sequentially from $k$ populations with densities belonging to an exponential family, in order to maximize the expected value of the sum $S_N = x_1 + \cdots + x_N$. These allocation rules are based on certain upper confidence bounds, which are developed from boundary crossing theory, for the $k$ population parameters. The rules are shown to be asymptotically optimal as $N \rightarrow \infty$ from both Bayesian and frequentist points of view. Monte Carlo studies show that they also perform very well for moderate values of the horizon $N$.

Herein we study a multivariate version of the adaptive stochastic approximation developed recently by Lai and Robbins. An adaptive procedure which involves a Venter-type estimate of the Jacobian of the response function is proposed and shown to be asymptotically efficient from both the estimation and the control points of view.

Stochastic procedures are randomized tests, estimates and confidence sets with two properties: (i) They are functions of an original sample and one or more artificially constructed auxiliary samples. (ii) They become nearly nonrandomized when the auxiliary samples increase in size. The stochastic procedures of this paper, which arise as approximations to numerically intractable procedures, involve iterated bootstrap techniques and random sampling schemes over abstract populations. A general methodology is applied to the asymptotic study of stochastic minimum distance tests, stochastic maximum likelihood estimates, stochastic confidence bands and several other stochastic procedures.

We establish rates for strong approximations of the bootstrapped product-limit process and the corresponding quantile process. These results are used to show weak convergence of bootstrapped total time on test and Lorenz curve processes to the same limiting Gaussian processes as for the unbootstrapped versions. We develop fully nonparametric confidence bands and tests for these curves and apply these results to prostate cancer. We also present almost sure results for the bootstrapped product-limit estimator.

We study the asymptotic properties of estimators of the tail of a distribution based on the excesses over a threshold. A key idea is the use of Pickands' generalised Pareto distribution and its fitting, in most cases, by the method of maximum likelihood. The results cover all three limiting types of extreme value theory. We propose a new estimator for an index of regular variation and show that it often performs better than Hill's estimator. We give new results for estimating the endpoint of a distribution, extending earlier work by Hall and by Smith and Weissman. Finally, we give detailed results for the domain of attraction of $\exp(-e^{-x})$ and show that, in most cases, our proposed estimator is more efficient than two others, one based on the exponential distribution and the other due to Davis and Resnick. We also touch briefly on the problem of large deviations from a statistical point of view. The results make extensive use of existing work on rates of convergence.

In this paper we introduce a partial ordering for positive dependent bivariate distributions. Our main result shows that tests of independence based on rank statistics such as Spearman's rho, Kendall's tau, Fisher-Yates' normal score statistic, van der Waerden's statistic and the quadrant statistic become more powerful under increasing positive dependence. In other words, these measures of positive dependence preserve the ordering stochastically in samples whenever it is present between underlying distributions.

To measure dependence in a set of random variables, a multivariate analog of maximal correlation is considered. This consists of transforming each of the variables so that the largest partial sums of the eigenvalues of the resulting correlation matrix is maximized. A "maximalized" measure of association obtained in this manner permits statements to be made about the strength of internal dependence exhibited by the random variables. It is shown, under a weak regularity condition, that optimizing transformations exist and that they satisfy a geometrically interpretable fixed point property. If the variables are jointly Gaussian, then the identity transformation is shown to be optimal, which extends Kolmogorov's result for canonical correlation to the principal components setting.

Polarization is used to unify and extend classical results concerning the completeness of generalized order statistics with dominated and nondominated distributions.

Consider the problem of estimating the common mean $\mu$ of two normal populations with unknown variances $\sigma^2_1$ and $\sigma^2_2$ under the quadratic loss $(\hat{\mu} - \mu)^2/\sigma^2_1$. A family of minimax estimators with smaller risk than the sample mean in the first population is given, out of which admissible minimax estimators are developed. A class of better estimators of $\mu$ under squared-error loss, which is wider than found by Bhattacharya, is obtained.

Consistency of Bayes procedures on the sphere is studied. Equivalent conditions for the consistency of Bayes procedures for a rotationally symmetric model are given. Equivalent conditions for that of an antipodally symmetric model are also provided.

It is shown under appropriate conditions that Rousseeuw's minimum volume estimator and other $S$-estimators of multivariate location and dispersion parameters are consistent. Under certain differentiability conditions the estimates are asymptotically normally distributed with a norming factor of $n^{1/2}$.

In this paper we generalize Hampel's concept of qualitative robustness of a sequence of estimators to the case of stochastic processes with non-i.i.d. observations, defining appropriate metrics between samples. We also present a different approach to qualitative robustness which formalizes the notion of resistance. We give two definitions based on this approach: strong and weak resistance. We show that for estimating a finite dimensional real parameter, $\pi$-robustness is equivalent to weak resistance and, in the i.i.d. case, is also equivalent to strong resistance. Finally, we prove the strong resistance of a class of estimators which includes common GM-estimates for linear models and autoregressive processes.

Let $\{X_n, n \geq 1\}$ be i.i.d with $P(X_i \leq u) = F(u)$ and $\{U_n, n \geq 1\}$ be i.i.d. with $P(U_i \leq u) = G(u). \hat{F}_n(t)$ is the Kaplan-Meier estimator based on the censored data $(\tilde{X}_i = X_i \wedge U_i, \delta_i = 1_{(X_i \leq U_i)}, 1 \leq i \leq n)$. In this note, it is shown that for $T_n = \max_{1 \leq i \leq n} \tilde{X}_i$, $\mathrm{pr}-\lim_{n \rightarrow \infty} \sup_{t \leq T_n} |\hat{F}_n(t) - F(t)| = 0.$ Hence, the largest interval on which the Kaplan-Meier estimator is uniformly consistent is found.

The Efron-Stein inequality is applied to prove that the kernel density estimate $f_n$, with an arbitrary nonnegative kernel and an arbitrary smoothing factor, satisfies the inequality $\operatorname{var}(\int|f_n - f|) \leq 4/n$ for all densities $f$. Similar inequalities are obtained for other estimates.

The problem of estimating the change-point in a sequence of independent random variables is considered. As the sample sizes before and after the change-point tend to infinity, Hinkley (1970) showed that the maximum likelihood estimate of the change-point converges in distribution to that of the change-point based on an infinite sample. Letting the amount of change in distribution approach 0, it is shown that the distribution, suitably normalized, of the maximum likelihood estimate based on an infinite sample converges to a simple one which is related to the location of the maximum for a two-sided Wiener process. Numerical results show that this simple distribution provides a good approximation to the exact distribution (with an infinite sample) in the normal case. However, it is unclear whether the approximation is good for general nonnormal cases.

Bather considered a buyer's problem and reduced it to an optimal stopping problem. In this note it is shown that this optimal stopping problem can be related to the sequential testing of the sign of the drift parameter of a Wiener process problem considered by Chernoff. Hence the asymptotic expansions for the sequential testing problem can be used in the buyer's problem.

The purpose of this note is to show that if a probability measure on a Euclidean space is not concentrated on a line, then its spatial median is unique.