Central to several objective approaches to Bayesian model selection is the use of training samples (subsets of the data), so as to allow utilization of improper objective priors. The most common prescription for choosing training samples is to choose them to be as small as possible, subject to yielding proper posteriors; these are called minimal training samples.

When data can vary widely in terms of either information content or impact on the improper priors, use of minimal training samples can be inadequate. Important examples include certain cases of discrete data, the presence of censored observations, and certain situations involving linear models and explanatory variables. Such situations require more sophisticated methods of choosing training samples. A variety of such methods are developed in this paper, and successfully applied in challenging situations.

Often the goal of model selection is to choose a model for future prediction, and it is natural to measure the accuracy of a future prediction by squared error loss. Under the Bayesian approach, it is commonly perceived that the optimal predictive model is the model with highest posterior probability, but this is not necessarily the case. In this paper we show that, for selection among normal linear models, the optimal predictive model is often the median probability model, which is defined as the model consisting of those variables which have overall posterior probability greater than or equal to 1/2 of being in a model. The median probability model often differs from the highest probability model.

This paper presents a model selection technique of estimation in semiparametric regression models of the type $Y_{i}={\beta}^{\prime}\underline{X}_{i}+f(T_{i})+W_{i}$ , i=1,…,n. The parametric and nonparametric components are estimated simultaneously by this procedure. Estimation is based on a collection of finite-dimensional models, using a penalized least squares criterion for selection. We show that by tailoring the penalty terms developed for nonparametric regression to semiparametric models, we can consistently estimate the subset of nonzero coefficients of the linear part. Moreover, the selected estimator of the linear component is asymptotically normal.

A class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by Fan and Li to simultaneously estimate parameters and select important variables. They demonstrated that this class of procedures has an oracle property when the number of parameters is finite. However, in most model selection problems the number of parameters should be large and grow with the sample size. In this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to ∞ as the sample size increases. Under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators. Furthermore, the consistency of the sandwich formula of the covariance matrix is demonstrated. Nonconcave penalized likelihood ratio statistics are discussed, and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions. The asymptotic results are augmented by a simulation study, and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary.

Higher criticism, or second-level significance testing, is a multiple-comparisons concept mentioned in passing by Tukey. It concerns a situation where there are many independent tests of significance and one is interested in rejecting the joint null hypothesis. Tukey suggested comparing the fraction of observed significances at a given α-level to the expected fraction under the joint null. In fact, he suggested standardizing the difference of the two quantities and forming a z-score; the resulting z-score tests the significance of the body of significance tests.

We consider a generalization, where we maximize this z-score over a range of significance levels 0<α≤α₀. We are able to show that the resulting higher criticism statistic is effective at resolving a very subtle testing problem: testing whether n normal means are all zero versus the alternative that a small fraction is nonzero.

The subtlety of this “sparse normal means” testing problem can be seen from work of Ingster and Jin, who studied such problems in great detail. In their studies, they identified an interesting range of cases where the small fraction of nonzero means is so small that the alternative hypothesis exhibits little noticeable effect on the distribution of the p-values either for the bulk of the tests or for the few most highly significant tests. In this range, when the amplitude of nonzero means is calibrated with the fraction of nonzero means, the likelihood ratio test for a precisely specified alternative would still succeed in separating the two hypotheses.

We show that the higher criticism is successful throughout the same region of amplitude sparsity where the likelihood ratio test would succeed. Since it does not require a specification of the alternative, this shows that higher criticism is in a sense optimally adaptive to unknown sparsity and size of the nonnull effects. While our theoretical work is largely asymptotic, we provide simulations in finite samples and suggest some possible applications. We also show that higher critcism works well over a range of non-Gaussian cases.

This paper discusses two goodness-of-fit testing problems. The first problem pertains to fitting an error distribution to an assumed nonlinear parametric regression model, while the second pertains to fitting a parametric regression model when the error distribution is unknown. For the first problem the paper contains tests based on a certain martingale type transform of residual empirical processes. The advantage of this transform is that the corresponding tests are asymptotically distribution free. For the second problem the proposed asymptotically distribution free tests are based on innovation martingale transforms. A Monte Carlo study shows that the simulated level of the proposed tests is close to the asymptotic level for moderate sample sizes.

This paper extends the theory of false discovery rates (FDR) pioneered by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300]. We develop a framework in which the False Discovery Proportion (FDP)—the number of false rejections divided by the number of rejections—is treated as a stochastic process. After obtaining the limiting distribution of the process, we demonstrate the validity of a class of procedures for controlling the False Discovery Rate (the expected FDP). We construct a confidence envelope for the whole FDP process. From these envelopes we derive confidence thresholds, for controlling the quantiles of the distribution of the FDP as well as controlling the number of false discoveries. We also investigate methods for estimating the p-value distribution.

We develop tests of the hypothesis of no effect for selected predictors in regression, without assuming a model for the conditional distribution of the response given the predictors. Predictor effects need not be limited to the mean function and smoothing is not required. The general approach is based on sufficient dimension reduction, the idea being to replace the predictor vector with a lower-dimensional version without loss of information on the regression. Methodology using sliced inverse regression is developed in detail.

This paper considers the problem of specifying a simple approximating density function for a given data set (x₁,…,x_n). Simplicity is measured by the number of modes but several different definitions of approximation are introduced. The taut string method is used to control the numbers of modes and to produce candidate approximating densities. Refinements are introduced that improve the local adaptivity of the procedures and the method is extended to spectral densities.

The concept of biased data is well known and its practical applications range from social sciences and biology to economics and quality control. These observations arise when a sampling procedure chooses an observation with probability that depends on the value of the observation. This is an interesting sampling procedure because it favors some observations and neglects others. It is known that biasing does not change rates of nonparametric density estimation, but no results are available about sharp constants. This article presents asymptotic results on sharp minimax density estimation. In particular, a coefficient of difficulty is introduced that shows the relationship between sample sizes of direct and biased samples that imply the same accuracy of estimation. The notion of the restricted local minimax, where a low-frequency part of the estimated density is known, is introduced; it sheds new light on the phenomenon of nonparametric superefficiency. Results of a numerical study are presented.

This article examines density estimation by combining a parametric approach with a nonparametric factor. The plug-in parametric estimator is seen as a crude estimator of the true density and is adjusted by a nonparametric factor. The nonparametric factor is derived by a criterion called local L₂-fitting. A class of estimators that have multiplicative adjustment is provided, including estimators proposed by several authors as special cases, and the asymptotic theories are developed. Theoretical comparison reveals that the estimators in this class are better than, or at least competitive with, the traditional kernel estimator in a broad class of densities. The asymptotically best estimator in this class can be obtained from the elegant feature of the bias function.

Empirical-likelihood-based confidence intervals for a mean were introduced by Owen [Biometrika 75 (1988) 237–249], where at least a finite second moment is required. This excludes some important distributions, for example, those in the domain of attraction of a stable law with index between 1 and 2. In this article we use a method similar to Qin and Wong [Scand. J. Statist. 23 (1996) 209–219] to derive an empirical-likelihood-based confidence interval for the mean when the underlying distribution has heavy tails. Our method can easily be extended to obtain a confidence interval for any order of moment of a heavy-tailed distribution.

This paper studies the least upper bounds on coverage probabilities of the empirical likelihood ratio confidence regions based on estimating equations. The implications of the bounds on empirical likelihood inference are also discussed.

We present the asymptotic distribution theory for a class of increment-based estimators of the fractal dimension of a random field of the form g{X(t)}, where g:R→R is an unknown smooth function and X(t) is a real-valued stationary Gaussian field on R^d, d=1 or 2, whose covariance function obeys a power law at the origin. The relevant theoretical framework here is “fixed domain” (or “infill”) asymptotics. Surprisingly, the limit theory in this non-Gaussian case is somewhat richer than in the Gaussian case (the latter is recovered when g is affine), in part because estimators of the type considered may have an asymptotic variance which is random in the limit. Broadly, when g is smooth and nonaffine, three types of limit distributions can arise, types (i), (ii) and (iii), say. Each type can be represented as a random integral. More specifically, type (i) can be represented as the integral of a certain random function with respect to Lebesgue measure; type (ii) can be represented as the integral of a second random function with respect to an independent Gaussian random measure; and type (iii) can be represented as a Wiener–Itô integral of order 2. Which type occurs depends on a combination of the following factors: the roughness of X(t), whether d=1 or d=2 and the order of the increment which is used. Another notable feature of our results is that, even though the estimators we consider are based on a variogram, no moment conditions are required on the observed field g{X(t)} for the limit theory to hold. The results of a numerical study are also presented.

We establish weak convergence of the empirical process on the spherical harmonics of a Gaussian random field in the presence of an unknown angular power spectrum. This result suggests various Gaussianity tests with an asymptotic justification. The issue of testing for Gaussianity on isotropic spherical random fields has recently received strong empirical attention in the cosmological literature, in connection with the statistical analysis of cosmic microwave background radiation.

A certain type of integer grid, called here an echelon grid, is an object found both in coherent systems whose components have a finite or countable number of levels and in algebraic geometry. If α=(α₁,…,α_d) is an integer vector representing the state of a system, then the corresponding algebraic object is a monomial x₁^α₁⋯x_d^α_d in the indeterminates x₁,…,x_d. The idea is to relate a coherent system to monomial ideals, so that the so-called Scarf complex of the monomial ideal yields an inclusion–exclusion identity for the probability of failure, which uses many fewer terms than the classical identity. Moreover in the “general position” case we obtain via the Scarf complex the tube bounds given by Naiman and Wynn [J. Inequal. Pure Appl. Math. (2001) 2 1–16]. Examples are given for the binary case but the full utility is for general multistate coherent systems and a comprehensive example is given.