For classification problems with significant class imbalance, subsampling can reduce computational costs at the price of inflated variance in estimating model parameters. We propose a method for subsampling efficiently for logistic regression by adjusting the class balance locally in feature space via an accept–reject scheme. Our method generalizes standard case-control sampling, using a pilot estimate to preferentially select examples whose responses are conditionally rare given their features. The biased subsampling is corrected by a post-hoc analytic adjustment to the parameters. The method is simple and requires one parallelizable scan over the full data set.

Standard case-control sampling is inconsistent under model misspecification for the population risk-minimizing coefficients $\theta^{*}$. By contrast, our estimator is consistent for $\theta^{*}$ provided that the pilot estimate is. Moreover, under correct specification and with a consistent, independent pilot estimate, our estimator has exactly twice the asymptotic variance of the full-sample MLE—even if the selected subsample comprises a miniscule fraction of the full data set, as happens when the original data are severely imbalanced. The factor of two improves to $1+\frac{1}{c}$ if we multiply the baseline acceptance probabilities by $c>1$ (and weight points with acceptance probability greater than 1), taking roughly $\frac{1+c}{2}$ times as many data points into the subsample. Experiments on simulated and real data show that our method can substantially outperform standard case-control subsampling.

Orthogonal array based space-filling designs (Owen [Statist. Sinica 2 (1992a) 439–452]; Tang [J. Amer. Statist. Assoc. 88 (1993) 1392–1397]) have become popular in computer experiments, numerical integration, stochastic optimization and uncertainty quantification. As improvements of ordinary Latin hypercube designs, these designs achieve stratification in multi-dimensions. If the underlying orthogonal array has strength $t$, such designs achieve uniformity up to $t$ dimensions. Existing central limit theorems are limited to these designs with only two-dimensional stratification based on strength two orthogonal arrays. We develop a new central limit theorem for these designs that possess stratification in arbitrary multi-dimensions associated with orthogonal arrays of general strength. This result is useful for building confidence statements for such designs in various statistical applications.

Variable selection, also known as feature selection in machine learning, plays an important role in modeling high dimensional data and is key to data-driven scientific discoveries. We consider here the problem of detecting influential variables under the general index model, in which the response is dependent of predictors through an unknown function of one or more linear combinations of them. Instead of building a predictive model of the response given combinations of predictors, we model the conditional distribution of predictors given the response. This inverse modeling perspective motivates us to propose a stepwise procedure based on likelihood-ratio tests, which is effective and computationally efficient in identifying important variables without specifying a parametric relationship between predictors and the response. For example, the proposed procedure is able to detect variables with pairwise, three-way or even higher-order interactions among $p$ predictors with a computational time of $O(p)$ instead of $O(p^{k})$ (with $k$ being the highest order of interactions). Its excellent empirical performance in comparison with existing methods is demonstrated through simulation studies as well as real data examples. Consistency of the variable selection procedure when both the number of predictors and the sample size go to infinity is established.

Modern construction of uniform confidence bands for nonparametric densities (and other functions) often relies on the classical Smirnov–Bickel–Rosenblatt (SBR) condition; see, for example, Giné and Nickl [Probab. Theory Related Fields 143 (2009) 569–596]. This condition requires the existence of a limit distribution of an extreme value type for the supremum of a studentized empirical process (equivalently, for the supremum of a Gaussian process with the same covariance function as that of the studentized empirical process). The principal contribution of this paper is to remove the need for this classical condition. We show that a considerably weaker sufficient condition is derived from an anti-concentration property of the supremum of the approximating Gaussian process, and we derive an inequality leading to such a property for separable Gaussian processes. We refer to the new condition as a generalized SBR condition. Our new result shows that the supremum does not concentrate too fast around any value.

We then apply this result to derive a Gaussian multiplier bootstrap procedure for constructing honest confidence bands for nonparametric density estimators (this result can be applied in other nonparametric problems as well). An essential advantage of our approach is that it applies generically even in those cases where the limit distribution of the supremum of the studentized empirical process does not exist (or is unknown). This is of particular importance in problems where resolution levels or other tuning parameters have been chosen in a data-driven fashion, which is needed for adaptive constructions of the confidence bands. Finally, of independent interest is our introduction of a new, practical version of Lepski’s method, which computes the optimal, nonconservative resolution levels via a Gaussian multiplier bootstrap method.

Ultra-high dimensional longitudinal data are increasingly common and the analysis is challenging both theoretically and methodologically. We offer a new automatic procedure for finding a sparse semivarying coefficient model, which is widely accepted for longitudinal data analysis. Our proposed method first reduces the number of covariates to a moderate order by employing a screening procedure, and then identifies both the varying and constant coefficients using a group SCAD estimator, which is subsequently refined by accounting for the within-subject correlation. The screening procedure is based on working independence and B-spline marginal models. Under weaker conditions than those in the literature, we show that with high probability only irrelevant variables will be screened out, and the number of selected variables can be bounded by a moderate order. This allows the desirable sparsity and oracle properties of the subsequent structure identification step. Note that existing methods require some kind of iterative screening in order to achieve this, thus they demand heavy computational effort and consistency is not guaranteed. The refined semivarying coefficient model employs profile least squares, local linear smoothing and nonparametric covariance estimation, and is semiparametric efficient. We also suggest ways to implement the proposed methods, and to select the tuning parameters. An extensive simulation study is summarized to demonstrate its finite sample performance and the yeast cell cycle data is analyzed.

We study the asymptotic behaviour of the posterior distribution in a broad class of statistical models where the “true” solution occurs on the boundary of the parameter space. We show that in this case Bayesian inference is consistent, and that the posterior distribution has not only Gaussian components as in the case of regular models (the Bernstein–von Mises theorem) but also has Gamma distribution components whose form depends on the behaviour of the prior distribution near the boundary and have a faster rate of convergence. We also demonstrate a remarkable property of Bayesian inference, that for some models, there appears to be no bound on efficiency of estimating the unknown parameter if it is on the boundary of the parameter space. We illustrate the results on a problem from emission tomography.

We analyze the performance of cross-validation (CV) in the density estimation framework with two purposes: (i) risk estimation and (ii) model selection. The main focus is given to the so-called leave-$p$-out CV procedure (Lpo), where $p$ denotes the cardinality of the test set. Closed-form expressions are settled for the Lpo estimator of the risk of projection estimators. These expressions provide a great improvement upon $V$-fold cross-validation in terms of variability and computational complexity.

From a theoretical point of view, closed-form expressions also enable to study the Lpo performance in terms of risk estimation. The optimality of leave-one-out (Loo), that is Lpo with $p=1$, is proved among CV procedures used for risk estimation. Two model selection frameworks are also considered: estimation, as opposed to identification. For estimation with finite sample size $n$, optimality is achieved for $p$ large enough [with $p/n=o(1)$] to balance the overfitting resulting from the structure of the model collection. For identification, model selection consistency is settled for Lpo as long as $p/n$ is conveniently related to the rate of convergence of the best estimator in the collection: (i) $p/n\to1$ as $n\to+\infty$ with a parametric rate, and (ii) $p/n=o(1)$ with some nonparametric estimators. These theoretical results are validated by simulation experiments.

We propose, for multivariate Gaussian copula models with unknown margins and structured correlation matrices, a rank-based, semiparametrically efficient estimator for the Euclidean copula parameter. This estimator is defined as a one-step update of a rank-based pilot estimator in the direction of the efficient influence function, which is calculated explicitly. Moreover, finite-dimensional algebraic conditions are given that completely characterize efficiency of the pseudo-likelihood estimator and adaptivity of the model with respect to the unknown marginal distributions. For correlation matrices structured according to a factor model, the pseudo-likelihood estimator turns out to be semiparametrically efficient. On the other hand, for Toeplitz correlation matrices, the asymptotic relative efficiency of the pseudo-likelihood estimator can be as low as 20%. These findings are confirmed by Monte Carlo simulations. We indicate how our results can be extended to joint regression models.

We continue the investigation of Bernstein–von Mises theorems for nonparametric Bayes procedures from [Ann. Statist. 41 (2013) 1999–2028]. We introduce multiscale spaces on which nonparametric priors and posteriors are naturally defined, and prove Bernstein–von Mises theorems for a variety of priors in the setting of Gaussian nonparametric regression and in the i.i.d. sampling model. From these results we deduce several applications where posterior-based inference coincides with efficient frequentist procedures, including Donsker– and Kolmogorov–Smirnov theorems for the random posterior cumulative distribution functions. We also show that multiscale posterior credible bands for the regression or density function are optimal frequentist confidence bands.

We consider the model of nonregular nonparametric regression where smoothness constraints are imposed on the regression function $f$ and the regression errors are assumed to decay with some sharpness level at their endpoints. The aim of this paper is to construct an adaptive estimator for the regression function $f$. In contrast to the standard model where local averaging is fruitful, the nonregular conditions require a substantial different treatment based on local extreme values. We study this model under the realistic setting in which both the smoothness degree $\beta>0$ and the sharpness degree $\mathfrak{a}\in(0,\infty)$ are unknown in advance. We construct adaptation procedures applying a nested version of Lepski’s method and the negative Hill estimator which show no loss in the convergence rates with respect to the general $L_{q}$-risk and a logarithmic loss with respect to the pointwise risk. Optimality of these rates is proved for $\mathfrak{a}\in(0,\infty)$. Some numerical simulations and an application to real data are provided.

Applying the Benjamini and Hochberg (B–H) method to multiple Student’s $t$ tests is a popular technique for gene selection in microarray data analysis. Given the nonnormality of the population, the true $p$-values of the hypothesis tests are typically unknown. Hence it is common to use the standard normal distribution $N(0,1)$, Student’s $t$ distribution $t_{n-1}$ or the bootstrap method to estimate the $p$-values. In this paper, we prove that when the population has the finite 4th moment and the dimension $m$ and the sample size $n$ satisfy $\log m=o(n^{1/3})$, the B–H method controls the false discovery rate (FDR) and the false discovery proportion (FDP) at a given level $\alpha$ asymptotically with $p$-values estimated from $N(0,1)$ or $t_{n-1}$ distribution. However, a phase transition phenomenon occurs when $\log m\geq c_{0}n^{1/3}$. In this case, the FDR and the FDP of the B–H method may be larger than $\alpha$ or even converge to one. In contrast, the bootstrap calibration is accurate for $\log m=o(n^{1/2})$ as long as the underlying distribution has the sub-Gaussian tails. However, such a light-tailed condition cannot generally be weakened. The simulation study shows that the bootstrap calibration is very conservative for the heavy tailed distributions. To solve this problem, a regularized bootstrap correction is proposed and is shown to be robust to the tails of the distributions. The simulation study shows that the regularized bootstrap method performs better than its usual counterpart.

In quantitative finance, we often model asset prices as a noisy Itô semimartingale. As this model is not identifiable, approximating by a time-changed Lévy process can be useful for generative modelling. We give a new estimate of the normalised volatility or time change in this model, which obtains minimax convergence rates, and is unaffected by infinite-variation jumps. In the semimartingale model, our estimate remains accurate for the normalised volatility, obtaining convergence rates as good as any previously implied in the literature.

Building on ideas from Castillo and Nickl [Ann. Statist. 41 (2013) 1999–2028], a method is provided to study nonparametric Bayesian posterior convergence rates when “strong” measures of distances, such as the sup-norm, are considered. In particular, we show that likelihood methods can achieve optimal minimax sup-norm rates in density estimation on the unit interval. The introduced methodology is used to prove that commonly used families of prior distributions on densities, namely log-density priors and dyadic random density histograms, can indeed achieve optimal sup-norm rates of convergence. New results are also derived in the Gaussian white noise model as a further illustration of the presented techniques.

We study the maximum smoothed likelihood estimator (MSLE) for interval censoring, case 2, in the so-called separated case. Characterizations in terms of convex duality conditions are given and strong consistency is proved. Moreover, we show that, under smoothness conditions on the underlying distributions and using the usual bandwidth choice in density estimation, the local convergence rate is $n^{-2/5}$ and the limit distribution is normal, in contrast with the rate $n^{-1/3}$ of the ordinary maximum likelihood estimator.