We develop a general approach to valid inference after model selection. At the core of our framework is a result that characterizes the distribution of a post-selection estimator conditioned on the selection event. We specialize the approach to model selection by the lasso to form valid confidence intervals for the selected coefficients and test whether all relevant variables have been included in the model.

We introduce nonparametric regularization of the eigenvalues of a sample covariance matrix through splitting of the data (NERCOME), and prove that NERCOME enjoys asymptotic optimal nonlinear shrinkage of eigenvalues with respect to the Frobenius norm. One advantage of NERCOME is its computational speed when the dimension is not too large. We prove that NERCOME is positive definite almost surely, as long as the true covariance matrix is so, even when the dimension is larger than the sample size. With respect to the Stein’s loss function, the inverse of our estimator is asymptotically the optimal precision matrix estimator. Asymptotic efficiency loss is defined through comparison with an ideal estimator, which assumed the knowledge of the true covariance matrix. We show that the asymptotic efficiency loss of NERCOME is almost surely 0 with a suitable split location of the data. We also show that all the aforementioned optimality holds for data with a factor structure. Our method avoids the need to first estimate any unknowns from a factor model, and directly gives the covariance or precision matrix estimator, which can be useful when factor analysis is not the ultimate goal. We compare the performance of our estimators with other methods through extensive simulations and real data analysis.

We establish global rates of convergence for the Maximum Likelihood Estimators (MLEs) of log-concave and $s$-concave densities on $\mathbb{R}$. The main finding is that the rate of convergence of the MLE in the Hellinger metric is no worse than $n^{-2/5}$ when $-1<s<\infty$ where $s=0$ corresponds to the log-concave case. We also show that the MLE does not exist for the classes of $s$-concave densities with $s<-1$.

Given an $n$-sample of random vectors $(X_{i},Y_{i})_{1\leq i\leq n}$ whose joint law is unknown, the long-standing problem of supervised classification aims to optimally predict the label $Y$ of a given new observation $X$. In this context, the $k$-nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations. Even if this algorithm is commonly used in the machine learning and statistics communities, less is known about its prediction ability in general finite dimensional spaces, especially when the support of the density of the observations is $\mathbb{R}^{d}$. This paper is devoted to the study of the statistical properties of the $k$-nearest neighbor rule in various situations. In particular, attention is paid to the marginal law of $X$, as well as the smoothness and margin properties of the regression function $\eta(X)=\mathbb{E}[Y|X]$. We identify two necessary and sufficient conditions to obtain uniform consistency rates of classification and derive sharp estimates in the case of the $k$-nearest neighbor rule. Some numerical experiments are proposed at the end of the paper to help illustrate the discussion.

In this paper, we first develop a new family of conjugate prior distributions for the cell probability parameters of discrete graphical models Markov with respect to a set $\mathcal{P}$ of moral directed acyclic graphs with skeleton a given decomposable graph $G$. This family, which we call the $\mathcal{P}$-Dirichlet, is a generalization of the hyper Dirichlet given in [Ann. Statist. 21 (1993) 1272–1317]: it keeps the directed strong hyper Markov property for every DAG in $\mathcal{P}$ but increases the flexibility in the choice of its parameters, that is, the hyper parameters.

Our second contribution is a characterization of the $\mathcal{P}$-Dirichlet, which yields, as a corollary, a characterization of the hyper Dirichlet and a characterization of the Dirichlet also. Like the characterization of the Dirichlet given in [Ann. Statist. 25 (1997) 1344–1369], our characterization of the $\mathcal{P}$-Dirichlet is based on local and global independence of the probability parameters and also a separability property explicitly defined here but implicitly used in that paper through the choice of two particular DAGs. Another advantage of our approach is that we need not make the assumption of the existence of a positive density function. We use the method of moments for our proofs.

We consider high-dimensional sparse regression problems in which we observe $\mathbf{y}=\mathbf{X}\boldsymbol{\beta} +\mathbf{z}$, where $\mathbf{X}$ is an $n\times p$ design matrix and $\mathbf{z}$ is an $n$-dimensional vector of independent Gaussian errors, each with variance $\sigma^{2}$. Our focus is on the recently introduced SLOPE estimator [Ann. Appl. Stat. 9 (2015) 1103–1140], which regularizes the least-squares estimates with the rank-dependent penalty $\sum_{1\le i\le p}\lambda_{i}|\widehat{\beta} |_{(i)}$, where $|\widehat{\beta} |_{(i)}$ is the $i$th largest magnitude of the fitted coefficients. Under Gaussian designs, where the entries of $\mathbf{X}$ are i.i.d. $\mathcal{N}(0,1/n)$, we show that SLOPE, with weights $\lambda_{i}$ just about equal to $\sigma\cdot\Phi^{-1}(1-iq/(2p))$ [$\Phi^{-1}(\alpha)$ is the $\alpha$th quantile of a standard normal and $q$ is a fixed number in $(0,1)$] achieves a squared error of estimation obeying \[\sup_{\|\boldsymbol{\beta} \|_{0}\le k}\mathbb{P} (\|\widehat{\boldsymbol {\beta}}_{\mathrm{SLOPE}}-\boldsymbol{\beta} \|^{2}>(1+\varepsilon) 2\sigma^{2}k\log(p/k))\longrightarrow 0\] as the dimension $p$ increases to $\infty$, and where $\varepsilon >0$ is an arbitrary small constant. This holds under a weak assumption on the $\ell_{0}$-sparsity level, namely, $k/p\rightarrow 0$ and $(k\log p)/n\rightarrow 0$, and is sharp in the sense that this is the best possible error any estimator can achieve. A remarkable feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of $\ell_{0}$-sparsity classes. We are not aware of any other estimator with this property.

In the setting of nonparametric multivariate regression with unknown error variance $\sigma^{2}$, we study asymptotic properties of a Bayesian method for estimating a regression function $f$ and its mixed partial derivatives. We use a random series of tensor product of B-splines with normal basis coefficients as a prior for $f$, and $\sigma$ is either estimated using the empirical Bayes approach or is endowed with a suitable prior in a hierarchical Bayes approach. We establish pointwise, $L_{2}$ and $L_{\infty}$-posterior contraction rates for $f$ and its mixed partial derivatives, and show that they coincide with the minimax rates. Our results cover even the anisotropic situation, where the true regression function may have different smoothness in different directions. Using the convergence bounds, we show that pointwise, $L_{2}$ and $L_{\infty}$-credible sets for $f$ and its mixed partial derivatives have guaranteed frequentist coverage with optimal size. New results on tensor products of B-splines are also obtained in the course.

We consider the optimal design problem for a comparison of two regression curves, which is used to establish the similarity between the dose response relationships of two groups. An optimal pair of designs minimizes the width of the confidence band for the difference between the two regression functions. Optimal design theory (equivalence theorems, efficiency bounds) is developed for this non-standard design problem and for some commonly used dose response models optimal designs are found explicitly. The results are illustrated in several examples modeling dose response relationships. It is demonstrated that the optimal pair of designs for the comparison of the regression curves is not the pair of the optimal designs for the individual models. In particular, it is shown that the use of the optimal designs proposed in this paper instead of commonly used “non-optimal” designs yields a reduction of the width of the confidence band by more than $50\%$.

We derive randomization-based models for experiments with a chain of randomizations. Estimation theory for these models leads to formulae for the estimators of treatment effects, their standard errors and expected mean squares in the analysis of variance. We discuss the practicalities in fitting these models and outline the difficulties that can occur, many of which do not arise in two-tiered experiments.

We propose a notion of conditional vector quantile function and a vector quantile regression. A conditional vector quantile function (CVQF) of a random vector $Y$, taking values in $\mathbb{R}^{d}$ given covariates $Z=z$, taking values in $\mathbb{R}^{k}$, is a map $u\longmapsto Q_{Y|Z}(u,z)$, which is monotone, in the sense of being a gradient of a convex function, and such that given that vector $U$ follows a reference non-atomic distribution $F_{U}$, for instance uniform distribution on a unit cube in $\mathbb{R}^{d}$, the random vector $Q_{Y|Z}(U,z)$ has the distribution of $Y$ conditional on $Z=z$. Moreover, we have a strong representation, $Y=Q_{Y|Z}(U,Z)$ almost surely, for some version of $U$. The vector quantile regression (VQR) is a linear model for CVQF of $Y$ given $Z$. Under correct specification, the notion produces strong representation, $Y=\beta (U)^{\top}f(Z)$, for $f(Z)$ denoting a known set of transformations of $Z$, where $u\longmapsto\beta(u)^{\top}f(Z)$ is a monotone map, the gradient of a convex function and the quantile regression coefficients $u\longmapsto\beta(u)$ have the interpretations analogous to that of the standard scalar quantile regression. As $f(Z)$ becomes a richer class of transformations of $Z$, the model becomes nonparametric, as in series modelling. A key property of VQR is the embedding of the classical Monge–Kantorovich’s optimal transportation problem at its core as a special case. In the classical case, where $Y$ is scalar, VQR reduces to a version of the classical QR, and CVQF reduces to the scalar conditional quantile function. An application to multiple Engel curve estimation is considered.

Panel data analysis is an important topic in statistics and econometrics. In such analysis, it is very common to assume the impact of a covariate on the response variable remains constant across all individuals. While the modelling based on this assumption is reasonable when only the global effect is of interest, in general, it may overlook some individual/subgroup attributes of the true covariate impact. In this paper, we propose a data driven approach to identify the groups in panel data with interactive effects induced by latent variables. It is assumed that the impact of a covariate is the same within each group, but different between the groups. An EM based algorithm is proposed to estimate the unknown parameters, and a binary segmentation based algorithm is proposed to detect the grouping. We then establish asymptotic theories to justify the proposed estimation, grouping method, and the modelling idea. Simulation studies are also conducted to compare the proposed method with the existing approaches, and the results obtained favour our method. Finally, the proposed method is applied to analyse a data set about income dynamics, which leads to some interesting findings.

Single index models offer greater flexibility in data analysis than linear models but retain some of the desirable properties such as the interpretability of the coefficients. We consider a pseudo-profile likelihood approach to estimation and testing for single-index quantile regression models. We establish the asymptotic normality of the index coefficient estimator as well as the optimal convergence rate of the nonparametric function estimation. Moreover, we propose a score test for the index coefficient based on the gradient of the pseudo-profile likelihood, and employ a penalized procedure to perform consistent model selection and model estimation simultaneously. We also use Monte Carlo studies to support our asymptotic results, and use an empirical example to illustrate the proposed method.

This paper provides a rigorous study of the nonparametric estimation of filaments or ridge lines of a probability density $f$. Points on the filament are considered as local extrema of the density when traversing the support of $f$ along the integral curve driven by the vector field of second eigenvectors of the Hessian of $f$. We “parametrize” points on the filaments by such integral curves, and thus both the estimation of integral curves and of filaments will be considered via a plug-in method using kernel density estimation. We establish rates of convergence and asymptotic distribution results for the estimation of both the integral curves and the filaments. The main theoretical result establishes the asymptotic distribution of the uniform deviation of the estimated filament from its theoretical counterpart. This result utilizes the extreme value behavior of nonstationary Gaussian processes indexed by manifolds $M_{h},h\in(0,1]$ as $h\to0$.

In this paper, we investigate frailty models for clustered survival data that are subject to both left- and right-censoring, termed “doubly-censored data”. This model extends current survival literature by broadening the application of frailty models from right-censoring to a more complicated situation with additional left-censoring.

Our approach is motivated by a recent Hepatitis B study where the sample consists of families. We adopt a likelihood approach that aims at the nonparametric maximum likelihood estimators (NPMLE). A new algorithm is proposed, which not only works well for clustered data but also improve over existing algorithm for independent and doubly-censored data, a special case when the frailty variable is a constant equal to one. This special case is well known to be a computational challenge due to the left-censoring feature of the data. The new algorithm not only resolves this challenge but also accommodates the additional frailty variable effectively.

Asymptotic properties of the NPMLE are established along with semi-parametric efficiency of the NPMLE for the finite-dimensional parameters. The consistency of Bootstrap estimators for the standard errors of the NPMLE is also discussed. We conducted some simulations to illustrate the numerical performance and robustness of the proposed algorithm, which is also applied to the Hepatitis B data.

In this paper, we study the approximation and estimation of $s$-concave densities via Rényi divergence. We first show that the approximation of a probability measure $Q$ by an $s$-concave density exists and is unique via the procedure of minimizing a divergence functional proposed by [Ann. Statist. 38 (2010) 2998–3027] if and only if $Q$ admits full-dimensional support and a first moment. We also show continuity of the divergence functional in $Q$: if $Q_{n}\to Q$ in the Wasserstein metric, then the projected densities converge in weighted $L_{1}$ metrics and uniformly on closed subsets of the continuity set of the limit. Moreover, directional derivatives of the projected densities also enjoy local uniform convergence. This contains both on-the-model and off-the-model situations, and entails strong consistency of the divergence estimator of an $s$-concave density under mild conditions. One interesting and important feature for the Rényi divergence estimator of an $s$-concave density is that the estimator is intrinsically related with the estimation of log-concave densities via maximum likelihood methods. In fact, we show that for $d=1$ at least, the Rényi divergence estimators for $s$-concave densities converge to the maximum likelihood estimator of a log-concave density as $s\nearrow0$. The Rényi divergence estimator shares similar characterizations as the MLE for log-concave distributions, which allows us to develop pointwise asymptotic distribution theory assuming that the underlying density is $s$-concave.

We correct an error in a technical lemma of Drees and Rootzén [Ann. Statist. 38 (2010) 2145–2186] and discuss consequences for applications.