We present a construction showing that a class of sets $\mathcal{C}$ that is Glivenko-Cantelli for an i.i.d. process need not be Glivenko-Cantelli for every stationary ergodic process with the same one dimensional marginal distribution. This result provides a counterpoint to recent work extending uniform strong laws to ergodic processes, and a recent characterization of universal Glivenko Cantelli classes.

The semiparametric normal copula model is studied with a correlation matrix that depends on a covariate. The bivariate version of this regression-copula model has been proposed for statistical analysis of Quantitative Trait Loci (QTL) via twin data. Appropriate linear combinations of Van der Waerden’s normal scores rank correlation coefficients yield $\sqrt{n}$-consistent estimators of the coefficients in the correlation function, i.e. of the regression parameters. They are used to construct semiparametrically efficient estimators of the regression parameters.

Stein’s [In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability 1 (1956) 197–206 University of California Press] asymptotically superior shrinkage estimator of $p$ univariate means may be rederived through adaptive penalized least squares (PLS) estimation in a regression model with one nominal covariate. This paper treats adaptive PLS estimators for $p$ unknown $d$-dimensional mean vectors, each of which depends on $k_{0}$ scalar covariates that may be nominal or ordinal. The initial focus is on complete regression designs, not necessarily balanced: for every $k_{0}$-tuple of possible covariate values, at least one observation is made on the corresponding mean vector. The results include definition of suitable candidate classes of PLS estimators in multivariate regression problems, comparison of these candidate estimators through their estimated quadratic risks under a general model that makes no assumptions about the regression function, and supporting asymptotic theory as the number $p$ of covariate-value combinations observed tends to infinity. Empirical process theory establishes that: (a) estimated risks converge to their intended targets uniformly over large classes of candidate PLS estimators; (b) a candidate PLS estimator that minimizes estimated risk within such a class has risk that converges to the minimal possible risk within the class. Extension of the results to incomplete regression designs is outlined. The Efron-Morris [ Journal of the American Statistical Association 68 (1973) 117–130] and Beran [ Annals of the Institute of Statistical Mathematics 60 (2008) 843–864] estimators for multivariate means in balanced complete designs are seen as special cases.

It was shown almost 40 years ago by Lucien Le Cam that the existence of suitable tests between Hellinger balls in the parameter set led to the construction of some sort of universal estimators for parametric statistical problems with i.i.d. observations. This idea of deriving estimators from families of robust tests was developed and substantially generalized in some of my previous work and more recently extended to Model Selection based estimation. Since the key ingredient for the design of such estimators for a given statistical framework is the construction of the relevant tests for this particular framework, it is essential to explain how to build them for as many different frameworks as possible. The purpose of this paper is to provide improved results about the existence of such tests for the problems of estimation based on independent (not necessarily i.i.d.) observations, estimation of conditional densities and of Markov transitions.

A standard estimation method when fitting parametric models to data from two-phase stratified samples is inverse probability weighting of the estimating equations. In previous work we applied this approach to likelihood equations for both Euclidean and non-Euclidean parameters in semi-parametric models. We proved weak convergence of the inverse probability weighted empirical process and derived an asymptotic expansion for the estimator of the Euclidean parameter. We also showed how adjustment of the sampling weights by their calibration to known totals of auxiliary variables, or their estimation using these same variables, could markedly improve efficiency.

Here we consider joint estimation of Euclidean and non-Euclidean parameters. Our asymptotic expansion for the non-Euclidean parameter is apparently new even in the special case of simple random sampling. The results are applied to estimation of survival probabilities for individual subjects using the regression coefficients (log hazard ratios) and baseline cumulative hazard function of the Cox proportional hazards model. Expressions derived for the variances of regression coefficients and cumulative hazards estimated after calibration of the weights aid construction of the auxiliary variables used for adjustment. We demonstrate empirically the improvement offered by calibration or estimation of the weights via simulation of two-phase stratified samples using publicly available data from the National Wilms Tumor Study and data analysis with the R survey package.

This paper introduces and analyzes a stochastic search method for parameter estimation in linear regression models in the spirit of Beran and Millar [ Ann. Statist. 15(3) (1987) 1131–1154]. The idea is to generate a random finite subset of a parameter space which will automatically contain points which are very close to an unknown true parameter. The motivation for this procedure comes from recent work of Dümbgen et al. [ Ann. Statist. 39(2) (2011) 702–730] on regression models with log-concave error distributions.

Let $P$ be a probability distribution on $q$-dimensional space. The so-called Diaconis-Freedman effect means that for a fixed dimension $d\ll q$, most $d$-dimensional projections of $P$ look like a scale mixture of spherically symmetric Gaussian distributions. The present paper provides necessary and sufficient conditions for this phenomenon in a suitable asymptotic framework with increasing dimension $q$. It turns out that the conditions formulated by Diaconis and Freedman [ Ann. Statist. 12 (1984) 793–815] are not only sufficient but necessary as well. Moreover, letting $\widehat{P}$ be the empirical distribution of $n$ independent random vectors with distribution $P$, we investigate the behavior of the empirical process $\sqrt{n}(\widehat{P}-P)$ under random projections, conditional on $\widehat{P}$.

We study asymptotically optimal statistical inference concerning the unknown state of $N$ identical quantum systems, using two complementary approaches: a “poor man’s approach” based on the van Trees inequality, and a rather more sophisticated approach using the recently developed quantum form of LeCam’s theory of Local Asymptotic Normality.

It is shown that the variable bandwidth density estimators proposed by McKay [ Canad. J. Statist. 21 (1993) 367–375; Variable kernel methods in density estimation (1993) Queen’s University] following earlier findings by Abramson [ Ann. Statist. 10 (1982) 1217–1223] approximate density functions in $C^4(\mathbb{R}^d)$ at the minimax rate in the supremum norm over bounded sets where the preliminary density estimates on which they are based are bounded away from zero. A somewhat more complicated estimator proposed by Jones, McKay and Hu [ Ann. Inst. Statist. Math. (1994) 46 521–535] to approximate densities in $C^6(\mathbb{R})$ can also be shown to attain minimax rates in sup norm over the same kind of sets. These estimators are strict probability densities.

Developing an effective multi-stage treatment strategy over time is one of the essential goals of modern medical research. Developing statistical inference, including constructing confidence intervals for parameters, is of key interest in studies applying dynamic treatment regimens. Estimation and inference in this context are especially challenging due to non-regularity caused by the non-smoothness of the problem in the parameters. While various bootstrap methods have been proposed, there is a lack of theoretical validation for most bootstrap inference methods. Recently, Song et al. [Penalized Q-learning for dynamic treatment regimes (2011) Submitted] proposed the penalized Q-learning procedure, that enables valid inference without the need of bootstrapping. As a major drawback, penalized Q-learning can only handle discrete covariates. To overcome this issue, we propose an adaptive Q-learning procedure which is an adaptive version of penalized Q-learning. We show that the proposed method can not only handle continuous covariates, but it can also be more efficient than penalized Q-learning.

A scoring function is consistent for the $\alpha$-quantile functional if, and only if, it is generalized piecewise linear (GPL) of order $\alpha$, up to equivalence. Expressed differently, loss functions that yield quantiles as Bayes rules are GPL functions. We review and discuss this basic decision-theoretic result with focus on Thomson’s pioneering characterization.

We discuss a number of estimates of the hazard under the assumption that the hazard is monotone on an interval $[0,a]$. The usual isotonic least squares estimators of the hazard are inconsistent at the boundary points $0$ and $a$. We use penalization to obtain uniformly consistent estimators. Moreover, we determine the optimal penalization constants, extending related work in this direction by [ Statist. Sinica 3 (1993) 501–515; Ann. Statist. 27 (1999) 338–360]. Two methods of obtaining smooth monotone estimates based on a non-smooth monotone estimator are discussed. One is based on kernel smoothing, the other on penalization.

We consider two variations on a Lehmann alternatives to symmetry-at-zero semiparametric model, with a real parameter $\theta$ quantifying skewness and a symmetric-at-0 distribution as a nuisance function. We show that a test of symmetry based on the signed log-rank statistic [A signed log-rank test of symmetry at zero (2011) University of Rochester] is asymptotically efficient in these models, derive its properties under local alternatives and present efficiency results relative to other signed-rank tests. We develop efficient estimation of the primary parameter in each model, using model-specific estimates of the nuisance function, and provide a method for choosing between the two models. All inference methods proposed are based solely on the signed ranks of the absolute values of the observations, the invariantly sufficient statistic. A simulation study is summarized and an example presented. Extensions to regression modeling are envisaged.

A problem of estimation of a large Hermitian nonnegatively definite matrix of trace 1 (a density matrix of a quantum system) motivated by quantum state tomography is studied. The estimator is based on a modified least squares method suitable in the case of models with random design with known design distributions. The bounds on Hilbert-Schmidt error of the estimator, including low rank oracle inequalities, have been proved. The proofs rely on Bernstein type inequalities for sums of independent random matrices.

We give bounds on the concentration of (pseudo) posterior distributions, both for correct and misspecified models. The bounds are derived using the information inequality, entropy estimates, and empirical process methods.

We use results from modern empirical process theory to establish a uniform in bandwidth central limit theorem, laws of the iterated logarithm and Glivenko–Cantelli theorem for kernel distribution function estimators.

Nemirovski’s inequality states that given independent and centered at expectation random vectors $X_{1},\ldots,X_{n}$ with values in $\ell^p(\mathbb{R}^d)$, there exists some constant $C(p,d)$ such that

\[\mathbb{E}\Vert S_n\Vert _p^2\le C(p,d)\sum_{i=1}^{n}\mathbb{E}\Vert X_i\Vert _p^2.\]

Furthermore $C(p,d)$ can be taken as $\kappa(p\wedge \log(d))$. Two cases were studied further in [ Am. Math. Mon. 117(2) (2010) 138–160]: general finite-dimensional Banach spaces and the special case $\ell^{\infty}(\mathbb{R}^{d})$. We show that in these two cases, it is possible to replace the quantity $\sum_{i=1}^n\mathbb{E}\Vert X_i\Vert _p^2$ by a smaller one without changing the order of magnitude of the constant when $d$ becomes large. In the spirit of [ Am. Math. Mon. 117(2) (2010) 138–160], our approach is probabilistic. The derivation of our version of Nemirovski’s inequality indeed relies on concentration inequalities.

Kagan and Shepp [ The American Statistician 59 (2005) 54–56] presented an elegant example of a mixture model for which an insufficient statistic preserves Fisher information. This note uses the regularity property of differentiability in quadratic mean to provide another explanation for the phenomenon they observed. Some connections with Le Cam’s theory for convergence of experiments are noted.

We consider the regression model with observation error in the design:

\begin{eqnarray*}y&=&X\theta^*+\xi,\\ Z&=&X+\Xi.\end{eqnarray*}

Here the random vector $y\in\mathbb{R}^n$ and the random $n\times p$ matrix $Z$ are observed, the $n\times p$ matrix $X$ is unknown, $\Xi$ is an $n\times p$ random noise matrix, $\xi\in\mathbb{R}^n$ is a random noise vector, and $\theta^*$ is a vector of unknown parameters to be estimated. We consider the setting where the dimension $p$ can be much larger than the sample size $n$ and $\theta^*$ is sparse. Because of the presence of the noise matrix $\Xi$, the commonly used Lasso and Dantzig selector are unstable. An alternative procedure called the Matrix Uncertainty (MU) selector has been proposed in Rosenbaum and Tsybakov [ The Annals of Statistics 38 (2010) 2620–2651] in order to account for the noise. The properties of the MU selector have been studied in Rosenbaum and Tsybakov [ The Annals of Statistics 38 (2010) 2620–2651] for sparse $\theta^*$ under the assumption that the noise matrix $\Xi$ is deterministic and its values are small. In this paper, we propose a modification of the MU selector when $\Xi$ is a random matrix with zero-mean entries having the variances that can be estimated. This is, for example, the case in the model where the entries of $X$ are missing at random. We show both theoretically and numerically that, under these conditions, the new estimator called the Compensated MU selector achieves better accuracy of estimation than the original MU selector.

The general asymptotic distribution theory for the functional regression model in Ruymgaart et al. [Some asymptotic theory for functional regression and classification (2009) Texas Tech University] simplifies considerably if an extra assumption on the random regressor is made. In the special case where the regressor is a stochastic process on the unit interval, Johannes [Privileged communication (2008)] assumes the regressor to be stationary, in which case the eigenfunctions of their covariance operator turn out to be known, so that only the eigenvalues are to be estimated. In the present paper we will also assume the eigenvectors to be known, but within an abstract setting. The simplification mentioned above is due to the circumstance that the covariance operator of the regressor commutes with its estimator as it can be constructed under the current conditions. Moreover, it is now possible to test linear hypotheses for the regression parameter that correspond to linear subspaces spanned by a finite number of the known eigenvectors.

We study high-dimensional linear models and the $\ell_1$-penalized least squares estimator, also known as the Lasso estimator. In literature, oracle inequalities have been derived under restricted eigenvalue or compatibility conditions. In this paper, we complement this with entropy conditions which allow one to improve the dual norm bound, and demonstrate how this leads to new oracle inequalities. The new oracle inequalities show that a smaller choice for the tuning parameter and a trade-off between $\ell_1$-norms and small compatibility constants are possible. This implies, in particular for correlated design, improved bounds for the prediction error of the Lasso estimator as compared to the methods based on restricted eigenvalue or compatibility conditions only.

Large-scale multiple testing problems require the simultaneous assessment of many p-values. This paper compares several methods to assess the evidence in multiple binomial counts of p-values: the maximum of the binomial counts after standardization (the “higher-criticism statistic”), the maximum of the binomial counts after a log-likelihood ratio transformation (the “Berk–Jones statistic”), and a newly introduced average of the binomial counts after a likelihood ratio transformation. Simulations show that the higher criticism statistic has a superior performance to the Berk–Jones statistic in the case of very sparse alternatives (sparsity coefficient $\beta \gtrapprox 0.75$), while the situation is reversed for $\beta \lessapprox 0.75$. The average likelihood ratio is found to combine the favorable performance of higher criticism in the very sparse case with that of the Berk–Jones statistic in the less sparse case and thus appears to dominate both statistics. Some asymptotic optimality theory is considered but found to set in too slowly to illuminate the above findings, at least for sample sizes up to one million. In contrast, asymptotic approximations to the critical values of the Berk–Jones statistic that have been developed by [In High Dimensional Probability III (2003) 321–332 Birkhäuser] and [ Ann. Statist. 35 (2007) 2018–2053] are found to give surprisingly accurate approximations even for quite small sample sizes.