The random numbers driving Markov chain Monte Carlo (MCMC) simulation are usually modeled as independent U(0, 1) random variables. Tribble [Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences (2007) Stanford Univ.] reports substantial improvements when those random numbers are replaced by carefully balanced inputs from completely uniformly distributed sequences. The previous theoretical justification for using anything other than i.i.d. U(0, 1) points shows consistency for estimated means, but only applies for discrete stationary distributions. We extend those results to some MCMC algorithms for continuous stationary distributions. The main motivation is the search for quasi-Monte Carlo versions of MCMC. As a side benefit, the results also establish consistency for the usual method of using pseudo-random numbers in place of random ones.

We study the approximation of arbitrary distributions P on d-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback–Leibler-type functional. We show that such an approximation exists if and only if P has finite first moments and is not supported by some hyperplane. Furthermore we show that this approximation depends continuously on P with respect to Mallows distance D₁(⋅, ⋅). This result implies consistency of the maximum likelihood estimator of a log-concave density under fairly general conditions. It also allows us to prove existence and consistency of estimators in regression models with a response Y=μ(X)+ε, where X and ε are independent, μ(⋅) belongs to a certain class of regression functions while ε is a random error with log-concave density and mean zero.

In high-dimensional linear regression, the goal pursued here is to estimate an unknown regression function using linear combinations of a suitable set of covariates. One of the key assumptions for the success of any statistical procedure in this setup is to assume that the linear combination is sparse in some sense, for example, that it involves only few covariates. We consider a general, nonnecessarily linear, regression with Gaussian noise and study a related question, that is, to find a linear combination of approximating functions, which is at the same time sparse and has small mean squared error (MSE). We introduce a new estimation procedure, called Exponential Screening, that shows remarkable adaptation properties. It adapts to the linear combination that optimally balances MSE and sparsity, whether the latter is measured in terms of the number of nonzero entries in the combination (ℓ₀ norm) or in terms of the global weight of the combination (ℓ₁ norm). The power of this adaptation result is illustrated by showing that Exponential Screening solves optimally and simultaneously all the problems of aggregation in Gaussian regression that have been discussed in the literature. Moreover, we show that the performance of the Exponential Screening estimator cannot be improved in a minimax sense, even if the optimal sparsity is known in advance. The theoretical and numerical superiority of Exponential Screening compared to state-of-the-art sparse procedures is also discussed.

We consider discrete-time observations of a continuous martingale under measurement error. This serves as a fundamental model for high-frequency data in finance, where an efficient price process is observed under microstructure noise. It is shown that this nonparametric model is in Le Cam’s sense asymptotically equivalent to a Gaussian shift experiment in terms of the square root of the volatility function σ and a nonstandard noise level. As an application, new rate-optimal estimators of the volatility function and simple efficient estimators of the integrated volatility are constructed.

In this paper, we study nonparametric estimation of the Lévy density for Lévy processes, with and without Brownian component. For this, we consider n discrete time observations with step Δ. The asymptotic framework is: n tends to infinity, Δ=Δ_n tends to zero while nΔ_n tends to infinity. We use a Fourier approach to construct an adaptive nonparametric estimator of the Lévy density and to provide a bound for the global ${\mathbb{L}}^{2}$-risk. Estimators of the drift and of the variance of the Gaussian component are also studied. We discuss rates of convergence and give examples and simulation results for processes fitting in our framework.

We define a conjugate prior for the reversible Markov chain of order r. The prior arises from a partially exchangeable reinforced random walk, in the same way that the Beta distribution arises from the exchangeable Polyá urn. An extension to variable-order Markov chains is also derived. We show the utility of this prior in testing the order and estimating the parameters of a reversible Markov model.

Structural equation models are multivariate statistical models that are defined by specifying noisy functional relationships among random variables. We consider the classical case of linear relationships and additive Gaussian noise terms. We give a necessary and sufficient condition for global identifiability of the model in terms of a mixed graph encoding the linear structural equations and the correlation structure of the error terms. Global identifiability is understood to mean injectivity of the parametrization of the model and is fundamental in particular for applicability of standard statistical methodology.

Suppose that we observe entries or, more generally, linear combinations of entries of an unknown m×T-matrix A corrupted by noise. We are particularly interested in the high-dimensional setting where the number mT of unknown entries can be much larger than the sample size N. Motivated by several applications, we consider estimation of matrix A under the assumption that it has small rank. This can be viewed as dimension reduction or sparsity assumption. In order to shrink toward a low-rank representation, we investigate penalized least squares estimators with a Schatten-p quasi-norm penalty term, p≤1. We study these estimators under two possible assumptions—a modified version of the restricted isometry condition and a uniform bound on the ratio “empirical norm induced by the sampling operator/Frobenius norm.” The main results are stated as nonasymptotic upper bounds on the prediction risk and on the Schatten-q risk of the estimators, where q∈[p, 2]. The rates that we obtain for the prediction risk are of the form rm/N (for m=T), up to logarithmic factors, where r is the rank of A. The particular examples of multi-task learning and matrix completion are worked out in detail. The proofs are based on tools from the theory of empirical processes. As a by-product, we derive bounds for the kth entropy numbers of the quasi-convex Schatten class embeddings S_p^M↪S₂^M, p<1, which are of independent interest.

The study of good nonregular fractional factorial designs has received significant attention over the last two decades. Recent research indicates that designs constructed from quaternary codes (QC) are very promising in this regard. The present paper shows how a trigonometric approach can facilitate a systematic understanding of such QC designs and lead to new theoretical results covering hitherto unexplored situations. We focus attention on one-eighth and one-sixteenth fractions of two-level factorials and show that optimal QC designs often have larger generalized resolution and projectivity than comparable regular designs. Moreover, some of these designs are found to have maximum projectivity among all designs.

We consider a two-stage procedure (TSP) for estimating an inverse regression function at a given point, where isotonic regression is used at stage one to obtain an initial estimate and a local linear approximation in the vicinity of this estimate is used at stage two. We establish that the convergence rate of the second-stage estimate can attain the parametric n^1/2 rate. Furthermore, a bootstrapped variant of TSP (BTSP) is introduced and its consistency properties studied. This variant manages to overcome the slow speed of the convergence in distribution and the estimation of the derivative of the regression function at the unknown target quantity. Finally, the finite sample performance of BTSP is studied through simulations and the method is illustrated on a data set.

We study the asymptotic behavior of kernel estimators of asymptotic variances (or long-run variances) for a class of adaptive Markov chains. The convergence is studied both in L^p and almost surely. The results also apply to Markov chains and improve on the existing literature by imposing weaker conditions. We illustrate the results with applications to the GARCH(1, 1) Markov model and to an adaptive MCMC algorithm for Bayesian logistic regression.

A general lower bound is developed for the minimax risk when estimating an arbitrary functional. The bound is based on testing two composite hypotheses and is shown to be effective in estimating the nonsmooth functional (1/n)∑|θ_i| from an observation Y∼N(θ, I_n). This problem exhibits some features that are significantly different from those that occur in estimating conventional smooth functionals. This is a setting where standard techniques fail to yield sharp results.

A sharp minimax lower bound is established by applying the general lower bound technique based on testing two composite hypotheses. A key step is the construction of two special priors and bounding the chi-square distance between two normal mixtures. An estimator is constructed using approximation theory and Hermite polynomials and is shown to be asymptotically sharp minimax when the means are bounded by a given value M. It is shown that the minimax risk equals β_∗²M²(log log n/log n)² asymptotically, where β_∗ is the Bernstein constant.

The general techniques and results developed in the present paper can also be used to solve other related problems.

We provide a new algorithm for the treatment of the deconvolution problem on the sphere which combines the traditional SVD inversion with an appropriate thresholding technique in a well chosen new basis. We establish upper bounds for the behavior of our procedure for any $\mathbb {L}_{p}$ loss. It is important to emphasize the adaptation properties of our procedures with respect to the regularity (sparsity) of the object to recover as well as to inhomogeneous smoothness. We also perform a numerical study which proves that the procedure shows very promising properties in practice as well.

We study an instance of high-dimensional inference in which the goal is to estimate a matrix Θ^∗∈ℝ^m₁×m₂ on the basis of N noisy observations. The unknown matrix Θ^∗ is assumed to be either exactly low rank, or “near” low-rank, meaning that it can be well-approximated by a matrix with low rank. We consider a standard M-estimator based on regularization by the nuclear or trace norm over matrices, and analyze its performance under high-dimensional scaling. We define the notion of restricted strong convexity (RSC) for the loss function, and use it to derive nonasymptotic bounds on the Frobenius norm error that hold for a general class of noisy observation models, and apply to both exactly low-rank and approximately low rank matrices. We then illustrate consequences of this general theory for a number of specific matrix models, including low-rank multivariate or multi-task regression, system identification in vector autoregressive processes and recovery of low-rank matrices from random projections. These results involve nonasymptotic random matrix theory to establish that the RSC condition holds, and to determine an appropriate choice of regularization parameter. Simulation results show excellent agreement with the high-dimensional scaling of the error predicted by our theory.

For planar landmark based shapes, taking into account the non-Euclidean geometry of the shape space, a statistical test for a common mean first geodesic principal component (GPC) is devised which rests on one of two asymptotic scenarios. For both scenarios, strong consistency and central limit theorems are established, along with an algorithm for the computation of a Ziezold mean geodesic. In application, this allows to verify the geodesic hypothesis for leaf growth of Canadian black poplars and to discriminate genetically different trees by observations of leaf shape growth over brief time intervals. With a test based on Procrustes tangent space coordinates, not involving the shape space’s curvature, neither can be achieved.

This paper focuses on generalizing quantiles from the ordering point of view. We propose the concept of partial quantiles, which are based on a given partial order. We establish that partial quantiles are equivariant under order-preserving transformations of the data, robust to outliers, characterize the probability distribution if the partial order is sufficiently rich, generalize the concept of efficient frontier, and can measure dispersion from the partial order perspective.

We also study several statistical aspects of partial quantiles. We provide estimators, associated rates of convergence, and asymptotic distributions that hold uniformly over a continuum of quantile indices. Furthermore, we provide procedures that can restore monotonicity properties that might have been disturbed by estimation error, establish computational complexity bounds, and point out a concentration of measure phenomenon (the latter under independence and the componentwise natural order).

Finally, we illustrate the concepts by discussing several theoretical examples and simulations. Empirical applications to compare intake nutrients within diets, to evaluate the performance of investment funds, and to study the impact of policies on tobacco awareness are also presented to illustrate the concepts and their use.

Because many illnesses show heterogeneous response to treatment, there is increasing interest in individualizing treatment to patients [Arch. Gen. Psychiatry 66 (2009) 128–133]. An individualized treatment rule is a decision rule that recommends treatment according to patient characteristics. We consider the use of clinical trial data in the construction of an individualized treatment rule leading to highest mean response. This is a difficult computational problem because the objective function is the expectation of a weighted indicator function that is nonconcave in the parameters. Furthermore, there are frequently many pretreatment variables that may or may not be useful in constructing an optimal individualized treatment rule, yet cost and interpretability considerations imply that only a few variables should be used by the individualized treatment rule. To address these challenges, we consider estimation based on l₁-penalized least squares. This approach is justified via a finite sample upper bound on the difference between the mean response due to the estimated individualized treatment rule and the mean response due to the optimal individualized treatment rule.

The delta method is a popular and elementary tool for deriving limiting distributions of transformed statistics, while applications of asymptotic distributions do not allow one to obtain desirable accuracy of approximation for tail probabilities. The large and moderate deviation theory can achieve this goal. Motivated by the delta method in weak convergence, a general delta method in large deviations is proposed. The new method can be widely applied to driving the moderate deviations of estimators and is illustrated by examples including the Wilcoxon statistic, the Kaplan–Meier estimator, the empirical quantile processes and the empirical copula function. We also improve the existing moderate deviations results for M-estimators and L-statistics by the new method. Some applications of moderate deviations to statistical hypothesis testing are provided.

In many social, economical, biological and medical studies, one objective is to classify a subject into one of several classes based on a set of variables observed from the subject. Because the probability distribution of the variables is usually unknown, the rule of classification is constructed using a training sample. The well-known linear discriminant analysis (LDA) works well for the situation where the number of variables used for classification is much smaller than the training sample size. Because of the advance in technologies, modern statistical studies often face classification problems with the number of variables much larger than the sample size, and the LDA may perform poorly. We explore when and why the LDA has poor performance and propose a sparse LDA that is asymptotically optimal under some sparsity conditions on the unknown parameters. For illustration of application, we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7,129 genes and a training sample of size 72. A simulation is also conducted to check the performance of the proposed method.

The celebrated de la Garza phenomenon states that for a polynomial regression model of degree p−1 any optimal design can be based on at most p design points. In a remarkable paper, Yang [Ann. Statist. 38 (2010) 2499–2524] showed that this phenomenon exists in many locally optimal design problems for nonlinear models. In the present note, we present a different view point on these findings using results about moment theory and Chebyshev systems. In particular, we show that this phenomenon occurs in an even larger class of models than considered so far.

We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model.

The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when n, the number of responses, and p, the number of predictors, stay bounded, and m, the number of observations, grows, but also when either, or both, n and p grow, possibly much faster than m.

We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term.

Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC. We offer a simple correction of the NNP estimator which leads to consistent rank estimation.

We verify and illustrate our theoretical findings via an extensive simulation study.

Supersaturated design (SSD) has received much recent interest because of its potential in factor screening experiments. In this paper, we provide equivalent conditions for two columns to be fully aliased and consequently propose methods for constructing E(f_NOD)- and χ²-optimal mixed-level SSDs without fully aliased columns, via equidistant designs and difference matrices. The methods can be easily performed and many new optimal mixed-level SSDs have been obtained. Furthermore, it is proved that the nonorthogonality between columns of the resulting design is well controlled by the source designs. A rather complete list of newly generated optimal mixed-level SSDs are tabulated for practical use.