ML-estimation based on mixtures of Normal distributions is a widely used tool for cluster analysis. However, a single outlier can make the parameter estimation of at least one of the mixture components break down. Among others, the estimation of mixtures of t-distributions by McLachlan and Peel [Finite Mixture Models (2000) Wiley, New York] and the addition of a further mixture component accounting for “noise” by Fraley and Raftery [The Computer J. 41 (1998) 578–588] were suggested as more robust alternatives. In this paper, the definition of an adequate robustness measure for cluster analysis is discussed and bounds for the breakdown points of the mentioned methods are given. It turns out that the two alternatives, while adding stability in the presence of outliers of moderate size, do not possess a substantially better breakdown behavior than estimation based on Normal mixtures. If the number of clusters s is treated as fixed, r additional points suffice for all three methods to let the parameters of r clusters explode. Only in the case of r=s is this not possible for t-mixtures. The ability to estimate the number of mixture components, for example, by use of the Bayesian information criterion of Schwarz [Ann. Statist. 6 (1978) 461–464], and to isolate gross outliers as clusters of one point, is crucial for an improved breakdown behavior of all three techniques. Furthermore, a mixture of Normals with an improper uniform distribution is proposed to achieve more robustness in the case of a fixed number of components.

In Bayesian decision theory, it is known that robustness with respect to the loss and the prior can be improved by adding new observations. In this article we study the rate of robustness improvement with respect to the number of observations n. Three usual measures of posterior global robustness are considered: the (range of the) Bayes actions set derived from a class of loss functions, the maximum regret of using a particular loss when the subjective loss belongs to a given class and the range of the posterior expected loss when the loss function ranges over a class. We show that the rate of convergence of the first measure of robustness is $\sqrt{n}$ , while it is n for the other measures under reasonable assumptions on the class of loss functions. We begin with the study of two particular cases to illustrate our results.

We describe and develop a close relationship between two problems that have customarily been regarded as distinct: that of maximizing entropy, and that of minimizing worst-case expected loss. Using a formulation grounded in the equilibrium theory of zero-sum games between Decision Maker and Nature, these two problems are shown to be dual to each other, the solution to each providing that to the other. Although Topsøe described this connection for the Shannon entropy over 20 years ago, it does not appear to be widely known even in that important special case.

We here generalize this theory to apply to arbitrary decision problems and loss functions. We indicate how an appropriate generalized definition of entropy can be associated with such a problem, and we show that, subject to certain regularity conditions, the above-mentioned duality continues to apply in this extended context. This simultaneously provides a possible rationale for maximizing entropy and a tool for finding robust Bayes acts. We also describe the essential identity between the problem of maximizing entropy and that of minimizing a related discrepancy or divergence between distributions. This leads to an extension, to arbitrary discrepancies, of a well-known minimax theorem for the case of Kullback–Leibler divergence (the “redundancy-capacity theorem” of information theory).

For the important case of families of distributions having certain mean values specified, we develop simple sufficient conditions and methods for identifying the desired solutions. We use this theory to introduce a new concept of “generalized exponential family” linked to the specific decision problem under consideration, and we demonstrate that this shares many of the properties of standard exponential families.

Finally, we show that the existence of an equilibrium in our game can be rephrased in terms of a “Pythagorean property” of the related divergence, thus generalizing previously announced results for Kullback–Leibler and Bregman divergences.

Most asymptotic results for robust estimates rely on regularity conditions that are difficult to verify in practice. Moreover, these results apply to fixed distribution functions. In the robustness context the distribution of the data remains largely unspecified and hence results that hold uniformly over a set of possible distribution functions are of theoretical and practical interest. Also, it is desirable to be able to determine the size of the set of distribution functions where the uniform properties hold. In this paper we study the problem of obtaining verifiable regularity conditions that suffice to yield uniform consistency and uniform asymptotic normality for location robust estimates when the scale of the errors is unknown. We study M-location estimates calculated with an S-scale and we obtain uniform asymptotic results over contamination neighborhoods. Moreover, we show how to calculate the maximum size of the contamination neighborhoods where these uniform results hold. There is a trade-off between the size of these neighborhoods and the breakdown point of the scale estimate.

We consider a class of semiparametric regression models which are one-parameter extensions of the Cox [J. Roy. Statist. Soc. Ser. B 34 (1972) 187–220] model for right-censored univariate failure times. These models assume that the hazard given the covariates and a random frailty unique to each individual has the proportional hazards form multiplied by the frailty. The frailty is assumed to have mean 1 within a known one-parameter family of distributions. Inference is based on a nonparametric likelihood. The behavior of the likelihood maximizer is studied under general conditions where the fitted model may be misspecified. The joint estimator of the regression and frailty parameters as well as the baseline hazard is shown to be uniformly consistent for the pseudo-value maximizing the asymptotic limit of the likelihood. Appropriately standardized, the estimator converges weakly to a Gaussian process. When the model is correctly specified, the procedure is semiparametric efficient, achieving the semiparametric information bound for all parameter components. It is also proved that the bootstrap gives valid inferences for all parameters, even under misspecification. We demonstrate analytically the importance of the robust inference in several examples. In a randomized clinical trial, a valid test of the treatment effect is possible when other prognostic factors and the frailty distribution are both misspecified. Under certain conditions on the covariates, the ratios of the regression parameters are still identifiable. The practical utility of the procedure is illustrated on a non-Hodgkin’s lymphoma dataset.

In the recent Bayesian nonparametric literature, many examples have been reported in which Bayesian estimators and posterior distributions do not achieve the optimal convergence rate, indicating that the Bernstein–von Mises theorem does not hold. In this article, we give a positive result in this direction by showing that the Bernstein–von Mises theorem holds in survival models for a large class of prior processes neutral to the right. We also show that, for an arbitrarily given convergence rate n^−α with 0<α≤1/2, a prior process neutral to the right can be chosen so that its posterior distribution achieves the convergence rate n^−α.

Thomas’ partial likelihood estimator of regression parameters is widely used in the analysis of nested case-control data with Cox’s model. This paper proposes a new estimator of the regression parameters, which is consistent and asymptotically normal. Its asymptotic variance is smaller than that of Thomas’ estimator away from the null. Unlike some other existing estimators, the proposed estimator does not rely on any more data than strictly necessary for Thomas’ estimator and is easily computable from a closed form estimating equation with a unique solution. The variance estimation is obtained as minus the inverse of the derivative of the estimating function and therefore the inference is easily available. A numerical example is provided in support of the theory.

One goal in survival analysis of right-censored data is to estimate the marginal survival function in the presence of dependent censoring. When many auxiliary covariates are sufficient to explain the dependent censoring, estimation based on either a semiparametric model or a nonparametric model of the conditional survival function can be problematic due to the high dimensionality of the auxiliary information. In this paper, we use two working models to condense these high-dimensional covariates in dimension reduction; then an estimate of the marginal survival function can be derived nonparametrically in a low-dimensional space. We show that such an estimator has the following double robust property: when either working model is correct, the estimator is consistent and asymptotically Gaussian; when both working models are correct, the asymptotic variance attains the efficiency bound.

The goal of this paper is to provide theorems on convergence rates of posterior distributions that can be applied to obtain good convergence rates in the context of density estimation as well as regression. We show how to choose priors so that the posterior distributions converge at the optimal rate without prior knowledge of the degree of smoothness of the density function or the regression function to be estimated.

An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density γ, with the mixing weight chosen by marginal maximum likelihood, in the hope of adapting between sparse and dense sequences. If estimation is then carried out using the posterior median, this is a random thresholding procedure. Other thresholding rules employing the same threshold can also be used. Probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n, allowing for sparsity of various kinds and degrees. The signal classes considered are “nearly black” sequences where only a proportion η is allowed to be nonzero, and sequences with normalized ℓ_p norm bounded by η, for η>0 and 0<p≤2. Estimation error is measured by mean qth power loss, for 0<q≤2. For all the classes considered, and for all q in (0,2], the method achieves the optimal estimation rate as n→∞ and η→0 at various rates, and in this sense adapts automatically to the sparseness or otherwise of the underlying signal. In addition the risk is uniformly bounded over all signals. If the posterior mean is used as the estimator, the results still hold for q>1. Simulations show excellent performance. For appropriately chosen functions γ, the method is computationally tractable and software is available. The extension to a modified thresholding method relevant to the estimation of very sparse sequences is also considered.

The purpose of this paper is to study optimality of circular neighbor-balanced block designs when neighbor effects are present in the model. In the literature many optimality results are established for direct effects and neighbor effects separately, but few for total effects, that is, the sum of direct effect of treatment and relevant neighbor effects. We show that circular neighbor-balanced designs are universally optimal for total effects among designs with no self neighbor. Then we give efficiency factors of these designs, and show some situations where a design with self neighbors is preferable to a neighbor-balanced design.

Booth and Cox proposed the E(s²) criterion for constructing two-level supersaturated designs. Nguyen [Technometrics 38 (1996) 69–73] and Tang and Wu [Canad. J. Statist 25 (1997) 191–201] independently derived a lower bound for E(s²). This lower bound can be achieved only when m is a multiple of N−1, where m is the number of factors and N is the run size. We present a method that uses difference families to construct designs that satisfy this lower bound. We also derive better lower bounds for the case where the Nguyen–Tang–Wu bound is not achievable. Our bounds cover more cases than a bound recently obtained by Butler, Mead, Eskridge and Gilmour [J. R. Stat. Soc. Ser. B Stat. Methodol. 63 (2001) 621–632]. New E(s²)-optimal designs are obtained by using a computer to search for designs that achieve the improved bounds.

In this article, model selection via penalized empirical loss minimization in nonparametric classification problems is studied. Data-dependent penalties are constructed, which are based on estimates of the complexity of a small subclass of each model class, containing only those functions with small empirical loss. The penalties are novel since those considered in the literature are typically based on the entire model class. Oracle inequalities using these penalties are established, and the advantage of the new penalties over those based on the complexity of the whole model class is demonstrated.

We study a simple learning algorithm for binary classification. Instead of predicting with the best hypothesis in the hypothesis class, that is, the hypothesis that minimizes the training error, our algorithm predicts with a weighted average of all hypotheses, weighted exponentially with respect to their training error. We show that the prediction of this algorithm is much more stable than the prediction of an algorithm that predicts with the best hypothesis. By allowing the algorithm to abstain from predicting on some examples, we show that the predictions it makes when it does not abstain are very reliable. Finally, we show that the probability that the algorithm abstains is comparable to the generalization error of the best hypothesis in the class.

The method of regularization with the Gaussian reproducing kernel is popular in the machine learning literature and successful in many practical applications. In this paper we consider the periodic version of the Gaussian kernel regularization. We show in the white noise model setting, that in function spaces of very smooth functions, such as the infinite-order Sobolev space and the space of analytic functions, the method under consideration is asymptotically minimax; in finite-order Sobolev spaces, the method is rate optimal, and the efficiency in terms of constant when compared with the minimax estimator is reasonably high. The smoothing parameters in the periodic Gaussian regularization can be chosen adaptively without loss of asymptotic efficiency. The results derived in this paper give a partial explanation of the success of the Gaussian reproducing kernel in practice. Simulations are carried out to study the finite sample properties of the periodic Gaussian regularization.

Simultaneous predictive distributions for independent Poisson observables are investigated. A class of improper prior distributions for Poisson means is introduced. The Bayesian predictive distributions based on priors from the introduced class are shown to be admissible under the Kullback–Leibler loss. A Bayesian predictive distribution based on a prior in this class dominates the Bayesian predictive distribution based on the Jeffreys prior.

We deal with the maximization of classical Fisher information in a quantum system depending on an unknown parameter. This problem has been raised by physicists, who defined [Helstrom (1967) Phys. Lett. A 25 101–102] a quantum counterpart of classical Fisher information, which has been found to constitute an upper bound for classical information itself [Braunstein and Caves (1994) Phys. Rev. Lett. 72 3439–3443]. It has then become of relevant interest among statisticians, who investigated the relations between classical and quantum information and derived a condition for equality in the particular case of two-dimensional pure state systems [Barndorff-Nielsen and Gill (2000) J. Phys. A 33 4481–4490].

In this paper we show that this condition holds even in the more general setting of two-dimensional mixed state systems. We also derive the expression of the maximum Fisher information achievable and its relation with that attainable in pure states.