Probability inequalities governing the approximation of the basic martingale, the rescaled difference between a counting process and its compensator, are derived. Applications to the random censoring model and to the field of goodness-of-fit tests are given. The inequalities are compared to inequalities derived earlier for the random censoring model using an empirical process approach.

In this paper, we develop a general theory for the convergence rate of sieve estimates, maximum likelihood estimates (MLE's) and related estimates obtained by optimizing certain empirical criteria in general parameter spaces. In many cases, especially when the parameter space is infinite dimensional, maximization over the whole parameter space is undesirable. In such cases, one has to perform maximization over an approximating space (sieve) of the original parameter space and allow the size of the approximating space to grow as the sample size increases. This method is called the method of sieves. In the case of the maximum likelihood estimation, an MLE based on a sieve is called a sieve MLE. We found that the convergence rate of a sieve estimate is governed by (a) the local expected values, variances and $L_2$ entropy of the criterion differences and (b) the approximation error of the sieve. A robust nonparametric regression problem, a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory. We also found that when the underlying space is too large, the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence, whereas the sieve estimate typically does not suffer from this difficulty.

A version of the multiple hypotheses testing problem is studied in which the decision procedure is based only on the current observation and the previous decision. Conditions for inconsistency and consistency of the stepwise Bayes rule, which are related to the boundedness of the likelihood ratios, are given. The (typically slow) rate of convergence of the error probabilities of consistent procedures is determined, and a sharp lower bound for the Bayes risk in terms of bounds on the likelihood ratios is derived. A modification of the recursive Sakrison's procedure for a continuous estimation problem is obtained in this setting by embedding the discrete family of original probability distributions into an exponential family.

This paper deals with Bayesian design for response surface prediction when the prior may be finite or infinite dimensional, the design space arbitrary. In order that the resulting problems be manageable, we resort to asymptotic versions of D-, G- and A-optimality. Here the asymptotics stem from allowing the error variance to be large. The problems thus elicited have strong game-like characteristics. Examples of theoretical solutions are brought forward, especially when the priors are stationary processes on an interval, and we give numerical evidence that the asymptotics work well in the finite domain.

Uncertainty in specification of the prior distribution is a common concern with Bayesian analysis. The robust Bayesian approach is to work with a class of prior distributions, which model uncertainty about the prior, instead of a single distribution. One is interested in the range of the posterior expectations of certain parametric functions as the prior varies over the class being considered--if this range is small, the analysis is robust to misspecification of the prior. Relatively little research has dealt with robustness with respect to priors on several parameters, especially the problem of imposing shape and smoothness constraints on the priors in the class. To address this problem, we consider neighborhood classes of mixture priors. Results are presented for two kinds of "mixture classes," which yield different types of neighborhoods. The problem of finding suprema and infima of posterior expectations of parametric functions is seen to reduce to numerical maximization and minimization. In the applications we consider mixtures of uniform densities on variously shaped sets. This allows one to model symmetry and unimodality of different types in more than one dimension. Numerical examples are provided.

Standard large-sample maximum likelihood and Bayesian inference, based on limiting multivariate normal distributions, may be dubious when applied with small or moderate sample sizes. We define and discuss several measures of nonnormality of MLE and posterior distributions that may be used as diagnostics and can indicate whether reparameterization will be effective in improving inferences. We begin by showing how the nonlinearity measures introduced by Beale and Bates and Watts for nonlinear regression may be generalized to exponential family nonlinear models. We replace the exponential family regression surface with another surface defined in terms of the parameterization in which the third derivatives of the loglikelihood function vanish at the MLE, and then we compute "curvatures" of the latter surface. This generalization effectively replaces the normal-theory Euclidean geometry with an $\alpha$-connection geometry of Amari identified by Kass, yet it may be understood and implemented without reference to that foundational argument. We also discuss alternative diagnostics based on the observed third derivatives of the loglikelihood function, or the third derivatives of a $\log$ posterior density. These may be viewed as multiparameter generalizations of a nonnormality measure proposed by Sprott. We show how one of these diagnostics may be quickly and easily computed using approximations of Tierney, Kass and Kadane.

In this article, a model of single variable sampling plan with type I censoring is studied. Under the assumption that the variable is exponentially distributed and the loss function is a polynomial, an explicit expression of the Bayes risk is derived. Then, a simple and finite algorithm to determine an optimal sampling plan for minimizing the Bayes risk is suggested. Furthermore, a discretization method is proposed so that one can easily obtain an approximately optimal sampling plan.

The frailty model is a generalization of Cox's proportional hazards model which includes a random effect. Nielsen, Gill, Andersen and Sorensen proposed an EM algorithm to estimate the cumulative baseline hazard and the variance of the random effect. Here existence and consistency of the estimators are proved. An example using truncated and censored data is considered.

The monotone class rank-test-based estimating equations for regression models with right censored data is considered. We introduce an estimator which is a solution of a monotone estimating equation that is an extension of the Gehan test. The estimator is easy to derive, $\sqrt n$-consistent and asymptotically normal under minimal conditions. All monotone estimating equations are characterized, and a simulation study, which shows that our suggested procedure performs well, is included.

We study two modified synthetic data least-squares estimation methods for linear regression models with right censored response variables, unspecified residual distributions and random censoring variables which may not be i.i.d. These methods are the result of an investigation into the use of stratification. We conclude that stratification should be used whether or not the censoring variables are dependent on the covariates. We give the asymptotic results of the estimators and numerical results.

Suppose that a system in its standard state produces i.i.d. observations whose distribution is symmetric about zero. At an unknown time the system may leave its standard state, and the observations would subsequently be stochastically larger. Subject to a bound on the rate of false alarms, one wants to detect quickly such a departure from the standard state. We present a robust method of detection which is computationally feasible and remarkably efficient. The method is based on the sequential vectors of signs and ranks of the observations. The methodology is one of likelihood ratio; a sequence of likelihood ratios for these vectors is computed, and the Shiryayev-Roberts approach to changepoint detection is then applied to yield a class of statistics and associated stopping rules. Inequalities and asymptotic approximations for the operating characteristics of these rules are developed. These are found to be valid also for small average run lengths and early changepoints as well. The relative efficiency of these schemes (with respect to a normal parametric shift detection policy) is very high, making them a robust alternative to parametric methods.

In this paper, a dynamic sampling plan in the Shiryayev-Roberts procedure is considered. It is shown that a two-rate dynamic sampling plan is optimal in the sense that it minimizes the stationary average delay time (SADT). Analytical results as well as numerical comparisons show that it is significantly superior to the fixed sampling plan. The comparison also shows that it is as powerful as the dynamic sampling procedure of Assaf and Ritov. The generalizations to the fast initial response and to the CUSUM procedure are also briefly discussed.

Observations $X_i$ are uncorrelated with means $\theta_i, i = 1,\cdots, n$, and variances 1. The linear estimators $\widehat{\theta} = T\mathbf{X}$, for some $n \times n$ matrix $T$, are widely used in smoothing problems, where it is assumed that neighbouring parameter values are similar. The smoothness assumption is violated in change point problems, where neighbouring parameter values are equal, except at some unspecified change points where there are jumps of unknown size from one parameter value to the next. In the case of a single change point in one dimension, for any linear estimator, the expected sum of squared errors between estimates and parameters is of order $\sqrt n$ for some choice of parameters, compared to order 1 for the least squares estimate. We show similar results for adaptive shift estimators, in which the linear estimator uses a kernel estimated from the data. Finally, for a change point problem in two dimensions, the expected sum of squared errors is of order $n^{3/4}$.

This paper deals with the following approach for estimating the mean $\mu$ of an $n$-dimensional random vector $Y$: first, a family $\mathbf{S}$ of $n \times n$ matrices is specified. Then, an element $\widehat{S} \in \mathbf{S}$ is selected by Mallows $C_L$, and $\widehat{\mu} = \widehat{S}\cdot Y$. The case is considered that $\mathbf{S}$ is an "ordered linear smoother" according to some easily interpretable, qualitative conditions. Examples include linear smoothing procedures in nonparametric regression (as, e.g., smoothing splines, minimax spline smoothers and kernel estimators). Stochastic probability bounds are given for the difference $(1/n)\|\mu - \widehat{S}\cdot Y\|^2_2 - (1/n)\|\mu - \widehat{S}_\mu\cdot Y\|^2_2$, where $\widehat{S}_\mu$ denotes the minimizer of $(1/n)\|\mu - S\cdot Y\|^2_2$ for $S \in \mathbf{S}$. These probability bounds are generalized to the situation that $\mathbf{S}$ is the union of a moderate number of ordered linear smoothers. The results complement work by Li on the asymptotic optimality of $C_L$. Implications for nonparametric regression are studied in detail. It is shown that there exists a direct connection between James-Stein estimation and the use of smoothing procedures, leading to a decision-theoretic justification of the latter. Further conclusions concern the choice of the order of a smoothing spline or a minimax spline smoother and the rates of convergence of smoothing parameters.

The local median regression method has long been known as a robustified alternative to methods such as local mean regression. Yet, its optimal statistical properties are largely unknown. In this paper, we show via decision-theoretic arguments that a local weighted median estimator is the best least absolute deviation estimator in an asymptotic minimax sense, under $L_1$-loss. We also study asymptotic efficiency of the local median estimator in the class of all possible estimators. From a practical viewpoint our results show that local weighted medians are preferable to histogram estimators, since they enjoy optimality properties which the latter do not, under virtually identical smoothness assumptions on the underlying curve. Among smoothing methods that are adapted to functions with only one derivative, little is to be gained by using an estimator other than one based on the local median.

We use the probabilistic method to show that if $f_{nh}$ is the standard kernel estimate with smoothing factor $h$, then there exists a deterministic sequence $h_n$ such that, for all densities, $\operatornamewithlimits{\lim\inf}_{n\rightarrow\infty} \frac{\mathbf{E} \int |f_{nh_n} - f|}{\inf_h \mathbf{E} \int |f_{nh} - f|} = 1.$

In the polynomial regression model of degree $m \in \mathbb{N}$ we consider the problem of determining a design for the identification of the correct degree of the underlying regression. We propose a new optimality criterion which minimizes a weighted $p$-mean of the variances of the least squares estimators for the coefficients of $x^l$ in the polynomial regression models of degree $l = 1,\cdots, m$. The theory of canonical moments is used to determine the optimal designs with respect to the proposed criterion. It is shown that the canonical moments of the optimal measure satisfy a (nonlinear) equation and that the support points are given by the zeros of an orthogonal polynomial. An explicit solution is given for the discrimination problem between polynomial regression models of degree $m - 2, m - 1$ and $m$ and the results are used to calculate the discrimination designs in the sense of Atkinson and Cox for polynomial regression models of degree $1,\cdots,m$.

In this paper we consider experimental situations where treatments from a mixed level factorial design are to be applied to experimental units over space or time and where there may be an unknown trend effect which can be approximated by some polynomial function of the order in which the observations are obtained. A method of constructing run orders of treatments for such situations is given which generally yields least squares estimators for main effects that have a higher degree of trend resistance than the least squares estimators of main effects that come from run orders obtainable by other methods of construction previously given in the literature.

Based on a duality between $E$-optimality for (sub-) parameters in weighted polynomial regression and a nonlinear approximation problem of Chebyshev type, in many cases the optimal approximate designs on nonnegative and nonpositive experimental regions $\lbrack a, b\rbrack$ are found to be supported by the extrema of the only equioscillating weighted polynomial over this region with leading coefficient 1. A similar result is stated for regression on symmetric regions $\lbrack -b, b\rbrack$ for certain subparameters, provided the region is "small enough," for example, $b \leq 1$. In particular, by specializing the weight function, we obtain results of Pukelsheim and Studden and of Dette.

Randomized orthogonal arrays provide good sets of input points for exploration of computer programs and for Monte Carlo integration. In 1954, Patterson gave a formula for the randomization variance of the sample mean of a function evaluated at the points of an orthogonal array. That formula is incorrect for most of the arrays that are practical for computer experiments. In this paper we correct Patterson's formula. We also remark on a defect, related to coincidences, in some orthogonal arrays. These are arrays of the form $OA(2q^2, 2q + 1, q, 2)$, where $q$ is a prime power, obtained by constructions due to Bose and Bush and to Addelman and Kempthorne. We conjecture that subarrays of the form $OA(2q^2, 2q, q, 2)$ may be constructed to avoid this defect.

We study linear combinations of order statistics ($L$-statistics) in survey problems with a stratified multistage sampling design. Two general types of $L$-statistics are considered: smooth $L$-statistics with weights generated by a smooth function and nonsmooth $L$-statistics (sample quantiles). The trimmed sample mean, the decile mean and variance, the sample Lorenz curve and the sample Gini's family parameters are examples of smooth $L$-statistics or functions of smooth $L$-statistics used in survey problems. It is shown that under weak conditions the smooth $L$-statistics are asymptotically normal and their asymptotic variances can be consistently estimated by jackknifing. For the sample quantiles, their asymptotic normality requires more conditions on the finite population distribution functions. Consistent estimators for the asymptotic variances of the sample quantiles are derived. Asymptotic validity of the Woodruff's confidence intervals for population quantiles is also proved.

In this paper, two versions of the Berry-Esseen theorems are established for $L$-statistics in the non-identically distributed case. One theorem, which requires $E|X_i|^3 < \infty$, is an extension of the classical Berry-Esseen theorem. Another, proved under the condition $E|X_i|^\alpha < \infty$ for some $\alpha \in (0, 1\rbrack$, seems to be of more interest for statistical inference. Some applications are also discussed.

We show that the empirical process of the block-based bootstrap observations from a stationary sequence converges weakly to an appropriate Gaussian process, conditionally in probability and almost surely, depending upon the block length. This bootstrap was introduced by Kunsch and later by Liu and Singh. Applications in estimation of the sampling distribution of a compactly differentiable functional are indicated.

We apply the bootstrap for general stationary observations, proposed by Kunsch, to the empirical process for $p$-dimensional random vectors. It is known that the empirical process in the multivariate case converges weakly to a certain Gaussian process. We show that the bootstrapped empirical process converges weakly to the same Gaussian process almost surely, assuming that the block length $l$ for constructing bootstrap replicates satisfies $l(n) = O(n^{1/2-\varepsilon}), 0 < \varepsilon < \frac{1}{2}$, and $l(n) \rightarrow \infty$. An example where the multivariate setup arises are the robust GM-estimates in an autoregressive model. We prove the asymptotic validity of the bootstrap approximation by showing that the functional associated with the GM-estimates is Frechet-differentiable.

This article considers a case of parametric bootstrap when the observations consist of generation sizes of the branching process with immigration together with the immigration component of each generation. Suppose we estimate the offspring mean $m$ by the maximum likelihood estimator (m.l.e.). It is then shown that the bootstrap version of the standardized m.l.e. does not have the same limiting distribution as the standardized m.l.e., under the assumption that $m = 1$ (critical case). In other words, the asymptotic validity does not hold for the parametric bootstrap in the critical case. In fact, given the sample, the value of the conditional distribution function of the bootstrap version of standardized m.l.e. defines a sequence of random variables whose limit (in distribution) is also shown to be a random variable, when $m = 1$. The approach used here is via a sequence of branching processes for which a general weak convergence [in $D^+\lbrack 0, \infty)$] result is established using operator semigroup convergence theorems.

Finite sample breakdown points are obtained for two classes of projection based multivariate location and scatter statistics: the Stahel-Donoho statistics and the Maronna-Yohai statistics. The definition of these multivariate statistics are dependent on the value of location and scale statistics for all univariate projections of the data, and consequently their properties depend on the nature of the corresponding univariate location and scale statistics used on the projected data. The finite sample breakdown points of the multivariate statistics, though, are not directly related to those of the corresponding univariate location and scale statistics. A uniform finite sample breakdown point concept is needed. The median and the median absolute deviation about the median (M.A.D.) are one possible choice for the univariate location and scale statistics, respectively. For sparse data sets in high dimensions, though, it is not recommended that the M.A.D. be used as the univariate scale statistic for the projected data since its uniform finite sample breakdown point is shown to be much less than optimum. A simple modification to the M.A.D., however, is shown to alleviate this problem. For various reasons, one may wish to consider univariate location and scale statistics other than the median and the M.A.D., respectively. A very broad and natural class of univariate location and scale statistics to consider for the projected data is the simultaneous $M$-estimates of location and scale. New results on their breakdown properties are given in this paper. Implicit formulas for the breakdown points of monotonic simultaneous $M$-estimates of location and scale are known, and they tend to imply rather low breakdown points for smooth choices of the defining weight functions. It is shown here that this phenomenon does not occur for a large class of nonmonotonic simultaneous $M$-estimates of location and scale. Furthermore, explicit rather than implicit expressions for the uniform finite sample breakdown points are given for these nonmonotonic $M$-estimates.

Under the usual assumptions of normality, the recursive estimator known as the Kalman filter gives excellent results and has found an extremely broad field of application--not only for estimating the state of a stochastic dynamic system, but also for estimating model parameters as well as detecting abrupt changes in the states or the parameters. It is well known, however, that significantly nonnormal noise, and particularly the presence of outliers, severely degrades the performance of the Kalman filter. This results in poor state estimates, nonwhite residuals and invalid inference. A first-order approximation is derived for the conditional prior distribution of the state of a discrete-time stochastic linear dynamic system in the presence of $\varepsilon$-contaminated normal observation noise. This distribution is then used to derive a first-order approximation of the conditional mean (minimum-variance) estimator. If the observation noise distribution can be represented as a member of the $\varepsilon$-contaminated normal neighborhood, then the conditional prior is also, to first order, an analogous perturbation from a normal distribution whose first two moments are given by the Kalman filter. Moreover, the perturbation is itself of a special form, combining distributions whose parameters are given by banks of parallel Kalman filters and optimal smoothers.

It is shown how and why the influence curve poorly measures the robustness properties of minimum Hellinger distance estimation. Rather, for this and related forms of estimation, there is another function, the residual adjustment function, that carries the relevant information about the trade-off between efficiency and robustness. It is demonstrated that this function determines various second-order measures of efficiency and robustness through a scalar measure called the estimation curvature. The function is also shown to determine the breakdown properties of the estimators through its tail behavior. A 50% breakdown result is given. It is shown how to create flexible classes of estimation methods in the spirit of $M$-estimation, but with first-order efficiency (or even second-order efficiency) at the chosen model, 50% breakdown and a minimum distance interpretation.