Gibbs samplers are popular algorithms to approximate posterior distributions arising from Bayesian hierarchical models. Despite their popularity and good empirical performance, however, there are still relatively few quantitative results on their convergence properties, for example, much less than for gradient-based sampling methods. In this work, we analyse the behaviour of total variation mixing times of Gibbs samplers targeting hierarchical models using tools from Bayesian asymptotics. We obtain dimension-free convergence results under random data-generating assumptions for a broad class of two-level models with generic likelihood function. Specific examples with Gaussian, binomial and categorical likelihoods are discussed.

Model-X approaches to testing conditional independence between a predictor and an outcome variable given a vector of covariates usually assume exact knowledge of the conditional distribution of the predictor given the covariates. Nevertheless, model-X methodologies are often deployed with this conditional distribution learned in sample. We investigate the consequences of this choice through the lens of the distilled conditional randomization test (dCRT). We find that Type-I error control is still possible, but only if the mean of the outcome variable given the covariates is estimated well enough. This demonstrates that the dCRT is doubly robust, and motivates a comparison to the generalized covariance measure (GCM) test, another doubly robust conditional independence test. We prove that these two tests are asymptotically equivalent, and show that the GCM test is optimal against (generalized) partially linear alternatives by leveraging semiparametric efficiency theory. In an extensive simulation study, we compare the dCRT to the GCM test. These two tests have broadly similar Type-I error and power, though dCRT can have somewhat better Type-I error control but somewhat worse power in small samples or when the response is discrete. We also find that post-lasso based test statistics (as compared to lasso based statistics) can dramatically improve Type-I error control for both methods.

The development of modern technology has enabled data collection of unprecedented size, which poses new challenges to many statistical estimation and inference problems. This paper studies the maximum score estimator of a semiparametric binary choice model under a distributed computing environment without prespecifying the noise distribution. An intuitive divide-and-conquer estimator is computationally expensive and restricted by a nonregular constraint on the number of machines, due to the highly nonsmooth nature of the objective function.

We propose (1) a one-shot divide-and-conquer estimator after smoothing the objective to relax the constraint, and (2) a multiround estimator to completely remove the constraint via iterative smoothing. We specify an adaptive choice of kernel smoother with a sequentially shrinking bandwidth to achieve the superlinear improvement of the optimization error over multiple iterations. The improved statistical accuracy per iteration is derived, and a quadratic convergence up to the optimal statistical error rate is established. We further provide two generalizations to handle the heterogeneity of data sets and high-dimensional problems where the parameter of interest is sparse.

Genetic prediction holds immense promise for translating genetic discoveries into medical advances. As the high-dimensional covariance matrix (or the linkage disequilibrium (LD) pattern) of genetic variants often presents a block-diagonal structure, numerous methods account for the dependence among variants in predetermined local LD blocks. Moreover, due to privacy considerations and data protection concerns, genetic variant dependence in each LD block is typically estimated from external reference panels rather than the original training data set. This paper presents a unified analysis of blockwise and reference panel-based estimators in a high-dimensional prediction framework without sparsity restrictions. We find that, surprisingly, even when the covariance matrix has a block-diagonal structure with well-defined boundaries, blockwise estimation methods adjusting for local dependence can be substantially less accurate than methods controlling for the whole covariance matrix. Further, estimation methods built on the original training data set and external reference panels are likely to have varying performance in high dimensions, which may reflect the cost of having only access to summary level data from the training data set. This analysis is based on novel results in random matrix theory for block-diagonal covariance matrix. We numerically evaluate our results using extensive simulations and real data analysis in the UK Biobank.

We analyze a number of natural estimators for the optimal transport map between two distributions and show that they are minimax optimal. We adopt the plugin approach: our estimators are simply optimal couplings between measures derived from our observations, appropriately extended so that they define functions on ${\mathbb{R}}^{\mathit{d}}$. When the underlying map is assumed to be Lipschitz, we show that computing the optimal coupling between the empirical measures, and extending it using linear smoothers, already gives a minimax optimal estimator. When the underlying map enjoys higher regularity, we show that the optimal coupling between appropriate nonparametric density estimates yields faster rates. Our work also provides new bounds on the risk of corresponding plugin estimators for the quadratic Wasserstein distance, and we show how this problem relates to that of estimating optimal transport maps using stability arguments for smooth and strongly convex Brenier potentials. As an application of our results, we derive central limit theorems for plugin estimators of the squared Wasserstein distance, which are centered at their population counterpart when the underlying distributions have sufficiently smooth densities. In contrast to known central limit theorems for empirical estimators, this result easily lends itself to statistical inference for the quadratic Wasserstein distance.

This paper concerns the limiting distributions of change-point estimators, in a high-dimensional linear regression time-series context, where a regression object $({\mathit{y}}_{\mathit{t}},{\mathit{X}}_{\mathit{t}})\in \mathbb{R}\times {\mathbb{R}}^{\mathit{p}}$ is observed at every time point $\mathit{t}\in \{1,\dots ,\mathit{n}\}$. At unknown time points, called change points, the regression coefficients change, with the jump sizes measured in ${\ell }_2$-norm. We provide limiting distributions of the change-point estimators in the regimes where the minimal jump size vanishes and where it remains a constant. We allow for both the covariate and noise sequences to be temporally dependent, in the functional dependence framework, which is the first time seen in the change-point inference literature. We show that a block-type long-run variance estimator is consistent under the functional dependence, which facilitates the practical implementation of our derived limiting distributions. We also present a few important byproducts of our analysis, which are of their own interest. These include a novel variant of the dynamic programming algorithm to boost the computational efficiency, consistent change-point localization rates under temporal dependence and a new Bernstein inequality for data possessing functional dependence. Extensive numerical results are provided to support our theoretical results. The proposed methods are implemented in the R package $\mathtt{changepoints}$ (Xu et al. (2022)).

We study the estimation of high-dimensional covariance matrices and their empirical spectral distributions under dynamic volatility models. Data under such models have nonlinear dependency both cross-sectionally and temporally. We establish the condition under which the limiting spectral distribution (LSD) of the sample covariance matrix under scalar BEKK models is different from the i.i.d. case. We then propose a time-variation adjusted (TV-adj) sample covariance matrix and prove that its LSD follows the Marčenko–Pastur law. Based on the asymptotics of the TV-adj sample covariance matrix, we develop a consistent population spectrum estimator and an asymptotically optimal nonlinear shrinkage estimator of the unconditional covariance matrix.

A novel sequential change detection problem is proposed, in which the goal is to not only detect but also accelerate the change. Specifically, it is assumed that the sequentially collected observations are responses to treatments selected in real time. The assigned treatments determine the pre-change and post-change distributions of the responses and also influence when the change happens. The goal is to find a treatment assignment rule and a stopping rule that minimize the expected total number of observations subject to a user-specified bound on the false alarm probability. The optimal solution is obtained under a general Markovian change-point model. Moreover, an alternative procedure is proposed, whose applicability is not restricted to Markovian change-point models and whose design requires minimal computation. For a large class of change-point models, the proposed procedure is shown to achieve the optimal performance in an asymptotic sense. Finally, its performance is found in simulation studies to be comparable to the optimal, uniformly with respect to the error probability.

Over the last decade, an approach that has gained a lot of popularity to tackle nonparametric testing problems on general (i.e., non-Euclidean) domains is based on the notion of reproducing kernel Hilbert space (RKHS) embedding of probability distributions. The main goal of our work is to understand the optimality of two-sample tests constructed based on this approach. First, we show the popular MMD (maximum mean discrepancy) two-sample test to be not optimal in terms of the separation boundary measured in Hellinger distance. Second, we propose a modification to the MMD test based on spectral regularization by taking into account the covariance information (which is not captured by the MMD test) and prove the proposed test to be minimax optimal with a smaller separation boundary than that achieved by the MMD test. Third, we propose an adaptive version of the above test which involves a data-driven strategy to choose the regularization parameter and show the adaptive test to be almost minimax optimal up to a logarithmic factor. Moreover, our results hold for the permutation variant of the test where the test threshold is chosen elegantly through the permutation of the samples. Through numerical experiments on synthetic and real data, we demonstrate the superior performance of the proposed test in comparison to the MMD test and other popular tests in the literature.

Multivariate adaptive regression splines (MARS) is a popular method for nonparametric regression introduced by Friedman in 1991. MARS fits simple nonlinear and non-additive functions to regression data. We propose and study a natural lasso variant of the MARS method. Our method is based on least squares estimation over a convex class of functions obtained by considering infinite-dimensional linear combinations of functions in the MARS basis and imposing a variation based complexity constraint. Our estimator can be computed via finite-dimensional convex optimization, although it is defined as a solution to an infinite-dimensional optimization problem. Under a few standard design assumptions, we prove that our estimator achieves a rate of convergence that depends only logarithmically on dimension and thus avoids the usual curse of dimensionality to some extent. We also show that our method is naturally connected to nonparametric estimation techniques based on smoothness constraints. We implement our method with a cross-validation scheme for the selection of the involved tuning parameter and compare it to the usual MARS method in various simulation and real data settings.

Within the nonparametric diffusion model, we develop a multiple test to infer about similarity of an unknown drift b to some reference drift ${\mathit{b}}_0$: At prescribed significance, we simultaneously identify those regions where violation from similarity occurs, without a priori knowledge of their number, size and location. This test is shown to be minimax-optimal and adaptive. At the same time, the procedure is robust under small deviation from Brownian motion as the driving noise process. A detailed investigation for fractional driving noise, which is neither a semimartingale nor a Markov process, is provided for Hurst indices close to the Brownian motion case.

Frequency domain analysis of time series is often difficult, as periodogram-based statistics involve non-linear averages with complicated variances. Due to the latter, nonparametric approximations from resampling or empirical likelihood (EL) are useful. However, current versions of periodogram-based EL for time series are highly restricted: these are valid only for linear processes and for special parameters (i.e., ratios). For general frequency domain inference with stationary, weakly dependent time series, we develop a spectral EL (SEL) method by combining two previously separate EL frameworks for time series: block-based EL and periodogram-based EL. This hybridization strategy is new and theoretically non-trivial, particularly as existing block-based EL relies on time domain averages that differ substantially from frequency domain counterparts. We formulate SEL statistics for parameters based on spectral estimating functions and periodogram subsamples. Under mild conditions, SEL log-ratio statistics are shown to be well-defined, admitting chi-square limits. Further, we formally establish an effective bootstrap procedure coupled with SEL. As a result, the SEL method can be used for nonparametric, asymptotically correct confidence regions and tests for frequency domain inference without explicit estimation of intricate variances of periodogram-based statistics. This broadly extends the applicability of EL for time series in three directions: (i) SEL can treat any spectral mean parameters; (ii) SEL is valid for both linear and non-linear processes; and (iii) SEL has a provable bootstrap development, which is rare for time series EL, and provides a novel alternative to other resampling approximations in the frequency domain. Simulation studies suggest the proposed method performs well compared to other non-EL approaches. A real data example demonstrates that SEL has application and extension to complicated scenarios.

We introduce a new nonparametric framework for classification problems in the presence of missing data. The key aspect of our framework is that the regression function decomposes into an anova-type sum of orthogonal functions, of which some (or even many) may be zero. Working under a general missingness setting, which allows features to be missing not at random, our main goal is to derive the minimax rate for the excess risk in this problem. In addition to the decomposition property, the rate depends on parameters that control the tail behaviour of the marginal feature distributions, the smoothness of the regression function and a margin condition. The ambient data dimension does not appear in the minimax rate, which can therefore be faster than in the classical nonparametric setting. We further propose a new method, called the Hard-thresholding Anova Missing data (HAM) classifier, based on a careful combination of a k-nearest neighbour algorithm and a thresholding step. The HAM classifier attains the minimax rate up to polylogarithmic factors and numerical experiments further illustrate its utility.

Linear sufficient dimension reduction, as exemplified by sliced inverse regression, has seen substantial development in the past thirty years. However, with the advent of more complex scenarios, nonlinear dimension reduction has gained considerable interest recently. This paper introduces a novel method for nonlinear sufficient dimension reduction, utilizing the generalized martingale difference divergence measure in conjunction with deep neural networks. The optimal solution of the proposed objective function is shown to be unbiased at the general level of σ-fields. And two optimization schemes, based on the fascinating deep neural networks, exhibit higher efficiency and flexibility compared to the classical eigendecomposition of linear operators. Moreover, we systematically investigate the slow rate and fast rate for the estimation error based on advanced U-process theory. Remarkably, the fast rate almost coincides with the minimax rate of nonparametric regression. The validity of our deep nonlinear sufficient dimension reduction methods is demonstrated through simulations and real data analysis.

Selective inference is the problem of giving valid answers to statistical questions chosen in a data-driven manner. A standard solution to selective inference is simultaneous inference, which delivers valid answers to the set of all questions that could possibly have been asked. However, simultaneous inference can be unnecessarily conservative if this set includes many questions that were unlikely to be asked in the first place. We introduce a less conservative solution to selective inference that we call locally simultaneous inference, which only answers those questions that could plausibly have been asked in light of the observed data, all the while preserving rigorous type I error guarantees. For example, if the objective is to construct a confidence interval for the “winning” treatment effect in a clinical trial with multiple treatments, and it is obvious in hindsight that only one treatment had a chance to win, then our approach will return an interval that is nearly the same as the uncorrected, standard interval. Locally simultaneous inference is implemented by refining any method for simultaneous inference of interest. Under mild conditions satisfied by common confidence intervals, locally simultaneous inference strictly dominates its underlying simultaneous inference method, meaning it can never yield less statistical power but only more. Compared to conditional selective inference, which demands stronger guarantees, locally simultaneous inference is more easily applicable in nonparametric settings and is more numerically stable.

Motivated by the statistical inference using the Gram matrix in the context of missing at random observations, this paper investigates the spectral properties of the random matrices ${\mathbf{S}}_{\mathit{n}}=\frac{1}{\mathit{n}}\mathbf{Z}{\mathbf{Z}}^{\ast }$, where $\mathbf{Z}=\mathbf{D}\circ ({\mathrm{\Sigma }}^{\mathbf{1}/\mathbf{2}}\mathbf{X})$ represents a Hadamard random matrix with entries determined by independent Bernoulli variables D. Operating within the high-dimensional framework, we establish the convergence of the empirical spectral distribution of ${\mathbf{S}}_{\mathit{n}}$ to a well-defined limiting distribution. In addition, we explore the impact of the missing mechanism on the second-order properties of the spectral distribution of the Gram matrix ${\mathbf{S}}_{\mathit{n}}$. We establish the central limit theorem for the linear spectral statistics of ${\mathbf{S}}_{\mathit{n}}$, shedding light on their fluctuations. Surprisingly, our analysis reveals that even in the ideal Gaussian distribution scenario, the fluctuations of statistics generated by eigenvalues are influenced by the eigenvectors of the population covariance matrix in the missing-at-random case. This discovery uncovers a remarkable phenomenon that starkly contrasts with the classical case. Subsequently, we demonstrate the practical application of our central limit theorem in hypothesis testing for the population covariance matrix.