We prove that boosting with the squared error loss, L₂Boosting, is consistent for very high-dimensional linear models, where the number of predictor variables is allowed to grow essentially as fast as O(exp(sample size)), assuming that the true underlying regression function is sparse in terms of the ℓ₁-norm of the regression coefficients. In the language of signal processing, this means consistency for de-noising using a strongly overcomplete dictionary if the underlying signal is sparse in terms of the ℓ₁-norm. We also propose here an AIC-based method for tuning, namely for choosing the number of boosting iterations. This makes L₂Boosting computationally attractive since it is not required to run the algorithm multiple times for cross-validation as commonly used so far. We demonstrate L₂Boosting for simulated data, in particular where the predictor dimension is large in comparison to sample size, and for a difficult tumor-classification problem with gene expression microarray data.

We attempt to recover an n-dimensional vector observed in white noise, where n is large and the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing power-law decay bounds on the ordered entries; and controlling the ℓ_p norm for p small. We obtain a procedure which is asymptotically minimax for ℓ^r loss, simultaneously throughout a range of such sparsity classes.

The optimal procedure is a data-adaptive thresholding scheme, driven by control of the false discovery rate (FDR). FDR control is a relatively recent innovation in simultaneous testing, ensuring that at most a certain expected fraction of the rejected null hypotheses will correspond to false rejections.

In our treatment, the FDR control parameter q_n also plays a determining role in asymptotic minimaxity. If q=lim q_n∈[0,1/2] and also q_n>γ/log(n), we get sharp asymptotic minimaxity, simultaneously, over a wide range of sparse parameter spaces and loss functions. On the other hand, q=lim q_n∈(1/2,1] forces the risk to exceed the minimax risk by a factor growing with q.

To our knowledge, this relation between ideas in simultaneous inference and asymptotic decision theory is new.

Our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors. These new rules impose complexity penalization of the form 2⋅log(potential model size/actual model sizes). We exhibit a close connection with FDR-controlling procedures under stringent control of the false discovery rate.

We consider covariate adjusted regression (CAR), a regression method for situations where predictors and response are observed after being distorted by a multiplicative factor. The distorting factors are unknown functions of an observable covariate, where one specific distorting function is associated with each predictor or response. The dependence of both response and predictors on the same confounding covariate may alter the underlying regression relation between undistorted but unobserved predictors and response. We consider a class of highly flexible adjustment methods for parameter estimation in the underlying regression model, which is the model of interest. Asymptotic normality of the estimates is obtained by establishing a connection to varying coefficient models. These distribution results combined with proposed consistent estimates of the asymptotic variance are used for the construction of asymptotic confidence intervals for the regression coefficients. The proposed approach is illustrated with data on serum creatinine, and finite sample properties of the proposed procedures are investigated through a simulation study.

Given an i.i.d. sample drawn from a density f, we propose to test that f equals some prescribed density f₀ or that f belongs to some translation/scale family. We introduce a multiple testing procedure based on an estimation of the $\mathbb{L}_{2}$-distance between f and f₀ or between f and the parametric family that we consider. For each sample size n, our test has level of significance α. In the case of simple hypotheses, we prove that our test is adaptive: it achieves the optimal rates of testing established by Ingster [J. Math. Sci. 99 (2000) 1110–1119] over various classes of smooth functions simultaneously. As for composite hypotheses, we obtain similar results up to a logarithmic factor. We carry out a simulation study to compare our procedures with the Kolmogorov–Smirnov tests, or with goodness-of-fit tests proposed by Bickel and Ritov [in Nonparametric Statistics and Related Topics (1992) 51–57] and by Kallenberg and Ledwina [Ann. Statist. 23 (1995) 1594–1608].

We introduce a new framework for constructing tests of general semiparametric hypotheses which have nontrivial power on the n^−1/2 scale in every direction, and can be tailored to put substantial power on alternatives of importance. The approach is based on combining test statistics based on stochastic processes of score statistics with bootstrap critical values.

We investigate the behavior of the nonparametric maximum likelihood estimator f̂_n for a decreasing density f near the boundaries of the support of f. We establish the limiting distribution of f̂_n(n^−α), where we need to distinguish between different values of 0<α<1. Similar results are obtained for the upper endpoint of the support, in the case it is finite. This yields consistent estimators for the values of f at the boundaries of the support. The limit distribution of these estimators is established and their performance is compared with the penalized maximum likelihood estimator.

The present paper investigates theoretical performance of various Bayesian wavelet shrinkage rules in a nonparametric regression model with i.i.d. errors which are not necessarily normally distributed. The main purpose is comparison of various Bayesian models in terms of their frequentist asymptotic optimality in Sobolev and Besov spaces.

We establish a relationship between hyperparameters, verify that the majority of Bayesian models studied so far achieve theoretical optimality, state which Bayesian models cannot achieve optimal convergence rate and explain why it happens.

We investigate shrinkage priors for constructing Bayesian predictive distributions. It is shown that there exist shrinkage predictive distributions asymptotically dominating Bayesian predictive distributions based on the Jeffreys prior or other vague priors if the model manifold satisfies some differential geometric conditions. Kullback–Leibler divergence from the true distribution to a predictive distribution is adopted as a loss function. Conformal transformations of model manifolds corresponding to vague priors are introduced. We show several examples where shrinkage predictive distributions dominate Bayesian predictive distributions based on vague priors.

A class of random hazard rates, which is defined as a mixture of an indicator kernel convolved with a completely random measure, is of interest. We provide an explicit characterization of the posterior distribution of this mixture hazard rate model via a finite mixture of S-paths. A closed and tractable Bayes estimator for the hazard rate is derived to be a finite sum over S-paths. The path characterization or the estimator is proved to be a Rao–Blackwellization of an existing partition characterization or partition-sum estimator. This accentuates the importance of S-paths in Bayesian modeling of monotone hazard rates. An efficient Markov chain Monte Carlo (MCMC) method is proposed to approximate this class of estimates. It is shown that S-path characterization also exists in modeling with covariates by a proportional hazard model, and the proposed algorithm again applies. Numerical results of the method are given to demonstrate its practicality and effectiveness.

We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution P₀, which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback–Leibler divergence with respect to P₀. An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

The iterative proportional fitting procedure (IPFP) was introduced formally by Deming and Stephan in 1940. For bivariate densities, this procedure has been investigated by Kullback and Rüschendorf. It is well known that the IPFP is a sequence of successive I-projections onto sets of probability measures with fixed marginals. However, when finding the I-projection onto the intersection of arbitrary closed, convex sets (e.g., marginal stochastic orders), a sequence of successive I-projections onto these sets may not lead to the actual solution. Addressing this situation, we present a new iterative I-projection algorithm. Under reasonable assumptions and using tools from Fenchel duality, convergence of this algorithm to the true solution is shown. The cases of infinite dimensional IPFP and marginal stochastic orders are worked out in this context.

The relationship between a time-dependent covariate and survival times is usually evaluated via the Cox model. Time-dependent covariates are generally available as longitudinal data collected regularly during the course of the study. A frequent problem, however, is the occurence of missing covariate data. A recent approach to estimation in the Cox model in this case jointly models survival and the longitudinal covariate. However, theoretical justification of this approach is still lacking. In this paper we prove existence and consistency of the maximum likelihood estimators in a joint model. The asymptotic distribution of the estimators is given along with a consistent estimator of the asymptotic variance.

A model for competing (resp. complementary) risks survival data where the failure time can be left (resp. right) censored is proposed. Product-limit estimators for the survival functions of the individual risks are derived. We deduce the strong convergence of our estimators on the whole real half-line without any additional assumptions and their asymptotic normality under conditions concerning only the observed distribution. When the observations are generated according to the double censoring model introduced by Turnbull, the product-limit estimators represent upper and lower bounds for Turnbull’s estimator.

Chain graphs (CG) (= adicyclic graphs) use undirected and directed edges to represent both structural and associative dependences. Like acyclic directed graphs (ADGs), the CG associated with a statistical Markov model may not be unique, so CGs fall into Markov equivalence classes, which may be superexponentially large, leading to unidentifiability and computational inefficiency in model search and selection. It is shown here that, under the Andersson–Madigan–Perlman (AMP) interpretation of a CG, each Markov-equivalence class can be uniquely represented by a single distinguished CG, the AMP essential graph, that is itself simultaneously Markov equivalent to all CGs in the AMP Markov equivalence class. A complete characterization of AMP essential graphs is obtained. Like the essential graph previously introduced for ADGs, the AMP essential graph will play a fundamental role for inference and model search and selection for AMP CG models.

We consider the finite-past predictor coefficients of stationary time series, and establish an explicit representation for them, in terms of the MA and AR coefficients. The proof is based on the alternate applications of projection operators associated with the infinite past and the infinite future. Applying the result to long memory processes, we give the rate of convergence of the finite predictor coefficients and prove an inequality of Baxter-type.

This paper addresses the problem of fitting a known distribution to the innovation distribution in a class of stationary and ergodic time series models. The asymptotic null distribution of the usual Kolmogorov–Smirnov test based on the residuals generally depends on the underlying model parameters and the error distribution. To overcome the dependence on the underlying model parameters, we propose that tests be based on a vector of certain weighted residual empirical processes. Under the null hypothesis and under minimal moment conditions, this vector of processes is shown to converge weakly to a vector of independent copies of a Gaussian process whose covariance function depends only on the fitted distribution and not on the model. Under certain local alternatives, the proposed test is shown to have nontrivial asymptotic power. The Monte Carlo critical values of this test are tabulated when fitting standard normal and double exponential distributions. The results obtained are shown to be applicable to GARCH and ARMA–GARCH models, the often used models in econometrics and finance. A simulation study shows that the test has satisfactory size and power for finite samples at these models. The paper also contains an asymptotic uniform expansion result for a general weighted residual empirical process useful in heteroscedastic models under minimal moment conditions, a result of independent interest.

In this paper we study strong approximations (invariance principles) of the sequential uniform and general Bahadur–Kiefer processes of long-range dependent sequences. We also investigate the strong and weak asymptotic behavior of the sequential Vervaat process, that is, the integrated sequential Bahadur–Kiefer process, properly normalized, as well as that of its deviation from its limiting process, the so-called Vervaat error process. It is well known that the Bahadur–Kiefer and the Vervaat error processes cannot converge weakly in the i.i.d. case. In contrast to this, we conclude that the Bahadur–Kiefer and Vervaat error processes, as well as their sequential versions, do converge weakly to a Dehling–Taqqu type limit process for certain long-range dependent sequences.