The Horseshoe is a widely used and popular continuous shrinkage prior for high-dimensional Bayesian linear regression. Recently, regularized versions of the Horseshoe prior have also been introduced in the literature. Various Gibbs sampling Markov chains have been developed in the literature to generate approximate samples from the corresponding intractable posterior densities. Establishing geometric ergodicity of these Markov chains provides crucial technical justification for the accuracy of asymptotic standard errors for Markov chain based estimates of posterior quantities. In this paper, we establish geometric ergodicity for various Gibbs samplers corresponding to the Horseshoe prior and its regularized variants in the context of linear regression. First, we establish geometric ergodicity of a Gibbs sampler for the original Horseshoe posterior under strictly weaker conditions than existing analyses in the literature. Second, we consider the regularized Horseshoe prior introduced in [18], and prove geometric ergodicity for a Gibbs sampling Markov chain to sample from the corresponding posterior without any truncation constraint on the global and local shrinkage parameters. Finally, we consider a variant of this regularized Horseshoe prior introduced in [15], and again establish geometric ergodicity for a Gibbs sampling Markov chain to sample from the corresponding posterior.

Although there is an extensive literature on the maxima of Gaussian processes, there are relatively few non-asymptotic bounds on their lower-tail probabilities. The aim of this paper is to develop such a bound, while also allowing for many types of dependence. Let $({\mathrm{\xi }}_1,\dots ,{\mathrm{\xi }}_N)$ be a centered Gaussian vector with standardized entries, whose correlation matrix R satisfies ${max}_{i\ne j}{R}_{ij}\le {\mathit{\rho }}_0$ for some constant ${\mathit{\rho }}_0\in (0,1)$. Then, for any ${\mathit{ϵ}}_0\in (0,\sqrt{1-{\mathit{\rho }}_0})$, we establish an upper bound on the probability $\mathbb{P}({max}_{1\le j\le N}{\mathrm{\xi }}_j\le {\mathit{ϵ}}_0\sqrt{2log(N)})$ in terms of $({\mathit{\rho }}_0,{\mathit{ϵ}}_0,N)$. The bound is also sharp, in the sense that it is attained up to a constant, independent of N. Next, we apply this result in the context of high-dimensional statistics, where we simplify and weaken conditions that have recently been used to establish near-parametric rates of bootstrap approximation. Lastly, an interesting aspect of this application is that it makes use of recent refinements of Bourgain and Tzafriri’s “restricted invertibility principle”.

Spectral singularities at non-zero frequencies play an important role in investigating cyclic or seasonal time series. The publication [2] introduced the generalized filtered method-of-moments approach to simultaneously estimate singularity location and long-memory parameters. This paper continues studies of these simultaneous estimators. A wide class of Gegenbauer-type semi-parametric models is considered. Asymptotic normality of several statistics of the cyclic and long-memory parameters is proved. New adjusted estimates are proposed and investigated. The theoretical findings are illustrated by numerical results. The methodology includes wavelet transformations as a particular case.

In this paper, we study a general class of causal processes with exogenous covariates, including many classical processes such as the ARMA-GARCH, APARCH, ARMAX, GARCH-X and APARCH-X processes. Under some Lipschitz-type conditions, the existence of a τ-weakly dependent strictly stationary and ergodic solution is established. We provide conditions for the strong consistency and derive the asymptotic distribution of the quasi-maximum likelihood estimator (QMLE), both when the true parameter is an interior point of the parameters space and when it belongs to the boundary. A significance Wald-type test of parameter is developed. This test is quite extensive and includes the test of nullity of the parameter’s components, which in particular, allows us to assess the relevance of the exogenous covariates. Relying on the QMLE of the model, we also propose a penalized criterion to address the problem of the model selection for this class. The weak and the strong consistency of the procedure are established. Finally, Monte Carlo simulations are conducted to numerically illustrate the main results.

The Bayesian approach to inverse problems with functional unknowns, has received significant attention in recent years. An important component of the developing theory is the study of the asymptotic performance of the posterior distribution in the frequentist setting. The present paper contributes to the area of Bayesian inverse problems by formulating a posterior contraction theory for linear inverse problems, with truncated Gaussian series priors, and under general smoothness assumptions. Emphasis is on the intrinsic role of the truncation point both for the direct as well as for the inverse problem, which are related through the modulus of continuity as this was recently highlighted by Knapik and Salomond (2018).

Huber loss, its asymmetric variants and their associated functionals (here named Huber functionals) are studied in the context of point forecasting and forecast evaluation. The Huber functional of a distribution is the set of minimizers of the expected (asymmetric) Huber loss, is an intermediary between a quantile and corresponding expectile, and also arises in M-estimation. Each Huber functional is elicitable, generating the precise set of minimizers of an expected score, subject to weak regularity conditions on the class of probability distributions, and has a complete characterization of its consistent scoring functions. Such scoring functions admit a mixture representation as a weighted average of elementary scoring functions. Each elementary score can be interpreted as the relative economic loss of using a particular forecast for a class of investment decisions where profits and losses are capped. The relevance of this theory for comparative assessment of weather forecasts is also discussed.

Random forests are among the most popular off-the-shelf supervised learning algorithms. Despite their well-documented empirical success, however, until recently, few theoretical results were available to describe their performance and behavior. In this work we push beyond recent work on consistency and asymptotic normality by establishing rates of convergence for random forests and other supervised learning ensembles. We develop the notion of generalized U-statistics and show that within this framework, random forest predictions can remain asymptotically normal for larger subsample sizes and under weaker conditions than previously established. Moreover, we provide Berry-Esseen bounds in order to quantify the rate at which this convergence occurs, making explicit the roles of the subsample size and the number of trees in determining the distribution of random forest predictions. When these generalized estimators are reduced to their classical U-statistic form, the rates we establish are faster than any available in the existing literature.

Bipartite networks are a natural representation of the interactions between entities from two different types. The organization (or topology) of such networks gives insight to understand the systems they describe as a whole. Here, we rely on motifs which provide a meso-scale description of the topology. Moreover, we consider the bipartite expected degree distribution (B-EDD) model which accounts for both the density of the network and possible imbalances between the degrees of the nodes. Under the B-EDD model, we prove the asymptotic normality of the count of any given motif, considering sparsity conditions. We also provide closed-form expressions for the mean and the variance of this count. This allows to avoid computationally prohibitive resampling procedures. Based on these results, we define a goodness-of-fit test for the B-EDD model and propose a family of tests for network comparisons. We assess the asymptotic normality of the test statistics and the power of the proposed tests on synthetic experiments and illustrate their use on ecological data sets.

Consider a random graph process with n vertices corresponding to points $v_i\stackrel{i.i.d.}{\sim }\mathrm{Unif}[0,1]$ embedded randomly in the interval, and where edges are inserted between $v_i,v_j$ independently with probability given by the graphon $w(v_i,v_j)\in [0,1]$. Following [11], we call a graphon w diagonally increasing if, for each x, $w(x,y)$ decreases as y moves away from x. We call a permutation $\mathit{\sigma }\in S_n$ an ordering of these vertices if $v_{\mathit{\sigma }(i)}<v_{\mathit{\sigma }(j)}$ for all $i<j$, and ask: how can we accurately estimate σ from an observed graph? We present a randomized algorithm with output $\hat{\mathit{\sigma }}$ that, for a large class of graphons, achieves error ${max}_{1\le i\le n}|\mathit{\sigma }(i)-\hat{\mathit{\sigma }}(i)|=\tilde{O}(\sqrt{n})$ with high probability; we also show that this is the best-possible convergence rate for a large class of algorithms and proof strategies. Under an additional assumption that is satisfied by some popular graphon models, we break this “barrier” at $\sqrt{n}$ and obtain the vastly better rate $\tilde{O}(n^{\mathit{ϵ}})$ for any $\mathit{ϵ}>0$. These improved seriation bounds can be combined with previous work to give more efficient and accurate algorithms for related tasks, including estimating diagonally increasing graphons [20, 21] and testing whether a graphon is diagonally increasing [11].

A solution manifold is the collection of points in a d-dimensional space satisfying a system of s equations with $s<d$. Solution manifolds occur in several statistical problems including missing data, algorithmic fairness, hypothesis testing, partial identifications, and nonparametric set estimation. We theoretically and algorithmically analyze solution manifolds. In terms of theory, we derive five useful results: smoothness theorem, stability theorem (which implies the consistency of a plug-in estimator), convergence of a gradient flow, local center manifold theorem and convergence of the gradient descent algorithm. We propose a Monte Carlo gradient descent algorithm to numerically approximate a solution manifold. In the case of the likelihood inference, we design a manifold constraint maximization procedure to find the maximum likelihood estimator on the manifold.

Screening is an important technique for analyzing high-dimensional data. Most screening tools have been developed for vectors and are marginal in the sense that each variable is evaluated individually at a time. Many multi-dimensional arrays (tensors) are generated nowadays. In addition to being high-dimensional, these data further have the tensor structure that should be exploited for more efficient analysis. Variables adjacent to each other in a tensor tend to be important or unimportant at the same time. Such information is ignored by marginal screening methods. In this article, we propose a general framework for tensor screening called smoothed tensor screening (STS). STS combines the strength of current marginal screening methods with tensor structural information by aggregating the information of its adjacent variables when evaluating one variable. STS is widely applicable since the statistical utility used in screening can be chosen based on the underlying model or data type of the responses and predictors. Moreover, we establish the SURE screening property for STS under mild conditions. Numerical studies demonstrate that STS has better performance than marginal screening methods.

For ultrahigh-dimensional data, variable screening is an important step to reduce the scale of the problem, hence, to improve the estimation accuracy and efficiency. In this paper, we propose a new dependence measure which is called the log odds ratio statistic to be used under the sufficient variable screening framework. The sufficient variable screening approach ensures the sufficiency of the selected input features in modeling the regression function and is an enhancement of existing marginal screening methods. In addition, we propose an ensemble variable screening approach to combine the proposed fused log odds ratio filter with the fused Kolmogorov filter to achieve supreme performance by taking advantages of both filters. We establish the sure screening properties of the fused log odds ratio filter for both marginal variable screening and sufficient variable screening. Extensive simulations and a real data analysis are provided to demonstrate the usefulness of the proposed log odds ratio filter and the sufficient variable screening procedure.

Statistical analysis of high-dimensional functional times series arises in various applications. Under this scenario, in addition to the intrinsic infinite-dimensionality of functional data, the number of functional variables can grow with the number of serially dependent observations. In this paper, we focus on the theoretical analysis of relevant estimated cross-(auto)covariance terms between two multivariate functional time series or a mixture of multivariate functional and scalar time series beyond the Gaussianity assumption. We introduce a new perspective on dependence by proposing functional cross-spectral stability measure to characterize the effect of dependence on these estimated cross terms, which are essential in the estimates for additive functional linear regressions. With the proposed functional cross-spectral stability measure, we develop useful concentration inequalities for estimated cross-(auto)covariance matrix functions to accommodate more general sub-Gaussian functional linear processes and, furthermore, establish finite sample theory for relevant estimated terms under a commonly adopted functional principal component analysis framework. Using our derived non-asymptotic results, we investigate the convergence properties of the regularized estimates for two additive functional linear regression applications under sparsity assumptions including functional linear lagged regression and partially functional linear regression in the context of high-dimensional functional/scalar time series.

We propose a fast and efficient strategy, called the representative approach, for big data analysis with generalized linear models, especially for distributed data with localization requirements or limited network bandwidth. With a given partition of massive dataset, this approach constructs a representative data point for each data block and fits the target model using the representative dataset. In terms of time complexity, it is as fast as the subsampling approaches in the literature. As for efficiency, its accuracy in estimating parameters given a homogeneous partition is comparable with the divide-and-conquer method. Supported by comprehensive simulation studies and theoretical justifications, we conclude that mean representatives (MR) work fine for linear models or generalized linear models with a flat inverse link function and moderate coefficients of continuous predictors. For general cases, we recommend the proposed score-matching representatives (SMR), which may improve the accuracy of estimators significantly by matching the score function values. As an illustrative application to the Airline on-time performance data, we show that the MR and SMR estimates are as good as the full data estimate when available.

Let $Z:=\{Z_t,t\ge 0\}$ be a stationary Gaussian process. We study two estimators of $\mathbb{E}[Z_0^2]$, namely ${\hat{f}}_T(Z):=\frac{1}{T}{\int }_0^T{Z}_t^{2}dt$, and ${\tilde{f}}_n(Z):=\frac{1}{n}{\sum }_{i=1}^n{Z}_{t_i}^2$, where $t_i=i{\mathrm{\Delta }}_n$, $i=0,1,\dots ,n$, ${\mathrm{\Delta }}_n\to 0$ and $T_n:=n{\mathrm{\Delta }}_n\to \mathrm{\infty }$. We prove that the two estimators are strongly consistent and establish Berry-Esseen bounds for a central limit theorem involving ${\hat{f}}_T(Z)$ and ${\tilde{f}}_n(Z)$. We apply these results to asymptotically stationary Gaussian processes and estimate the drift parameter for Gaussian Ornstein-Uhlenbeck processes.

We introduce Markov Random Geometric Graphs (MRGGs), a growth model for temporal dynamic networks. It is based on a Markovian latent space dynamic: consecutive latent points are sampled on the Euclidean Sphere using an unknown Markov kernel; and two nodes are connected with a probability depending on a unknown function of their latent geodesic distance.

More precisely, at each stamp-time k we add a latent point $X_k$ sampled by jumping from the previous one $X_{k-1}$ in a direction chosen uniformly $Y_k$ and with a length $r_k$ drawn from an unknown distribution called the latitude function. The connection probabilities between each pair of nodes are equal to the envelope function of the distance between these two latent points. We provide theoretical guarantees for the non-parametric estimation of the latitude and the envelope functions.

We propose an efficient algorithm that achieves those non-parametric estimation tasks based on an ad-hoc Hierarchical Agglomerative Clustering approach. As a by product, we show how MRGGs can be used to detect dependence structure in growing graphs and to solve link prediction problems.

The infinite-dimensional Hilbert sphere ${\mathcal{S}}^{\mathrm{\infty }}$ has been widely employed to model density functions and shapes, extending the finite-dimensional counterpart. We consider the Fréchet mean as an intrinsic summary of the central tendency of data lying on ${\mathcal{S}}^{\mathrm{\infty }}$. For sound statistical inference, we derive properties of the Fréchet mean on ${\mathcal{S}}^{\mathrm{\infty }}$ by establishing its existence and uniqueness as well as a root-n central limit theorem (CLT) for the sample version, overcoming obstructions from infinite-dimensionality and lack of compactness on ${\mathcal{S}}^{\mathrm{\infty }}$. Intrinsic CLTs for the estimated tangent vectors and covariance operator are also obtained. Asymptotic and bootstrap hypothesis tests for the Fréchet mean based on projection and norm are then proposed and are shown to be consistent. The proposed two-sample tests are applied to make inference for daily taxi demand patterns over Manhattan, modeled as densities, of which the square root densities are analyzed on the Hilbert sphere. Numerical properties of the proposed hypothesis tests which utilize the spherical geometry are studied in the real data application and simulations, where we demonstrate that the tests based on the intrinsic geometry compare favorably to those based on an extrinsic or flat geometry.

Precision medicine is an increasingly important area of research. Due to the heterogeneity of individual characteristics, patients may respond differently to treatments. One of the most important goals for precision medicine is to develop individualized treatment rules (ITRs) involving patients’ characteristics directly. As an interesting topic in clinical research, many statistical methods have been developed in recent years to find optimal ITRs. For binary treatments, outcome weighted learning (OWL) was proposed to find a decision function of patient characteristics maximizing the expected clinical outcome. Treatments with hierarchical structure are commonly seen in practice. In hierarchical scenarios, how to estimate ITRs is still unclear. We propose a new framework named hierarchical outcome-weighted angle-based learning (HOAL) to estimate ITRs for treatments with hierarchical structure. Statistical properties including Fisher consistency and convergence rates of the proposed method are presented. Simulations and an application to a type 2 diabetes study under linear and nonlinear learning show the highly competitive performance of our proposed procedure in both numerical accuracy and computational efficiency.

In this paper, we provide a general methodology to draw statistical inferences on individual signal coordinates or linear combinations of them in sparse phase retrieval. Given an initial estimator of the targeting parameter, which is generated by some existing algorithm, we can modify it in a way that the modified version is asymptotically normal and unbiased. Then confidence intervals and hypothesis testing can be constructed based on this asymptotic normality. For conciseness, we focus on confidence intervals in this work, while a similar procedure can be adopted for hypothesis testing. Under some mild assumptions on the signal and sample size, we establish theoretical guarantees for the proposed method. These assumptions are generally weak in the sense that the dimension could exceed the sample size and many non-zero small coordinates are allowed. Furthermore, theoretical analysis reveals that the modified estimators for individual coordinates have uniformly bounded variance, and hence simultaneous inference is possible. Numerical simulations in a wide range of settings are supportive of our theoretical results.

In binary classification, imbalance refers to situations in which one class is heavily under-represented. This issue is due to either a data collection process or because one class is indeed rare in a population. Imbalanced classification frequently arises in applications such as biology, medicine, engineering, and social sciences. In this paper, for the first time, we theoretically study the impact of imbalance class sizes on the linear discriminant analysis (LDA) in high dimensions. We show that due to data scarcity in one class, referred to as the minority class, and high-dimensionality of the feature space, the LDA ignores the minority class yielding a maximum misclassification rate. We then propose a new construction of hard-thresholding rules based on a data splitting technique that reduces the large difference between the misclassification rates. We show that the proposed method is asymptotically optimal. We further study two well-known sparse versions of the LDA in imbalanced cases. We evaluate the finite-sample performance of different methods using simulations and by analyzing two real data sets. The results show that our method either outperforms its competitors or has comparable performance based on a much smaller subset of selected features, while being computationally more efficient.

We investigate the large-sample behavior of change-point tests based on weighted two-sample U-statistics, in the case of short-range dependent data. Under some mild mixing conditions, we establish convergence of the test statistic to an extreme value distribution. A simulation study shows that the weighted tests are superior to the non-weighted versions when the change-point occurs near the boundary of the time interval, while they loose power in the center.

In this paper we present new theoretical results on optimal estimation of certain random quantities based on high frequency observations of a Lévy process. More specifically, we investigate the asymptotic theory for the conditional mean and conditional median estimators of the supremum/infimum of a linear Brownian motion and a strictly stable Lévy process. Another contribution of our article is the conditional mean estimation of the local time and the occupation time of a linear Brownian motion. We demonstrate that the new estimators are considerably more efficient compared to the classical estimators studied in e.g. [6, 14, 29, 30, 38]. Furthermore, we discuss pre-estimation of the parameters of the underlying models, which is required for practical implementation of the proposed statistics.

Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani [49]. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.

Current status censoring (CSC) implies that there is no direct access to the lifetime of an event of interest. Instead it is known if the event already occurred or not at a random monitoring time. CSC is a simple sampling procedure and in many cases the only possibility to assess the lifetime of interest. At the same time, the absence of a direct measurement of a lifetime of interest makes the problem of nonparametric distribution estimation ill-posed. A simple, adaptive and sharp minimax estimator of the density and cumulative distribution function is proposed. The simplicity of estimator also allows us to relax assumptions. Practical examples illustrate CSC problem and the proposed estimator.

We introduce two new bootstraps for exchangeable random graphs. One, the “empirical graphon bootstrap”, is based purely on resampling, while the other, the “histogram bootstrap”, is a model-based “sieve” bootstrap. We show that both of them accurately approximate the sampling distributions of motif densities, i.e., of the normalized counts of the number of times fixed subgraphs appear in the network. These densities characterize the distribution of (infinite) exchangeable networks. Our bootstraps therefore give a valid quantification of uncertainty in inferences about fundamental network statistics, and so of parameters identifiable from them.

We consider the problem of change point detection for high-dimensional distributions in a location family when the dimension can be much larger than the sample size. In change point analysis, the widely used cumulative sum (CUSUM) statistics are sensitive to outliers and heavy-tailed distributions. In this paper, we propose a robust, tuning-free (i.e., fully data-dependent), and easy-to-implement change point test that enjoys strong theoretical guarantees. To achieve the robust purpose in a nonparametric setting, we formulate the change point detection in the multivariate U-statistics framework with anti-symmetric and nonlinear kernels. Specifically, the within-sample noise is canceled out by anti-symmetry of the kernel, while the signal distortion under certain nonlinear kernels can be controlled such that the between-sample change point signal is magnitude preserving. A (half) jackknife multiplier bootstrap (JMB) tailored to the change point detection setting is proposed to calibrate the distribution of our ${\ell }^{\mathrm{\infty }}$-norm aggregated test statistic. Subject to mild moment conditions on kernels, we derive the uniform rates of convergence for the JMB to approximate the sampling distribution of the test statistic, and analyze its size and power properties. Extensions to multiple change point testing and estimation are discussed with illustration from numerical studies.

This paper presents a number of new findings about the canonical change point estimation problem. The first part studies the estimation of a change point on the real line in a simple stump model using the robust Huber estimating function which interpolates between the ${\ell }_1$ (absolute deviation) and ${\ell }_2$ (least squares) based criteria. While the ${\ell }_2$ criterion has been studied extensively, its robust counterparts and in particular, the ${\ell }_1$ minimization problem have not. We derive the limit distribution of the estimated change point under the Huber estimating function and compare it to that under the ${\ell }_2$ criterion. Theoretical and empirical studies indicate that it is more profitable to use the Huber estimating function (and in particular, the ${\ell }_1$ criterion) under heavy tailed errors as it leads to smaller asymptotic confidence intervals at the usual levels compared to the ${\ell }_2$ criterion. We also compare the ${\ell }_1$ and ${\ell }_2$ approaches in a parallel setting, where one has m independent single change point problems and the goal is to control the maximal deviation of the estimated change points from the true values, and establish rigorously that the ${\ell }_1$ estimation criterion provides a superior rate of convergence to the ${\ell }_2$, and that this relative advantage is driven by the heaviness of the tail of the error distribution. Finally, we derive minimax optimal rates for the change plane estimation problem in growing dimensions and demonstrate that Huber estimation attains the optimal rate while the ${\ell }_2$ scheme produces a rate sub-optimal estimator for heavy tailed errors. In the process of deriving our results, we establish a number of properties about the minimizers of compound Binomial and compound Poisson processes which are of independent interest.

We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it is sub-optimal for the regression coefficient; that is, a minimax estimator for the class of bandable covariances may not be a minimax estimator for its functionals. We propose the blockwise tapering estimator of the regression coefficient, which has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We also propose a Bayesian procedure called the blockwise tapering post-processed posterior of the regression coefficient and show that the proposed Bayesian procedure has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We show that the proposed methods outperform the existing methods via numerical studies.

From the distributional characterizations that lie at the heart of Stein’s method we derive explicit formulae for the mass functions of discrete probability laws that identify those distributions. These identities are applied to develop tools for the solution of statistical problems. Our characterizations, and hence the applications built on them, do not require any knowledge about normalization constants of the probability laws. To demonstrate that our statistical methods are sound, we provide comparative simulation studies for the testing of fit to the Poisson distribution and for parameter estimation of the negative binomial family when both parameters are unknown. We also consider the problem of parameter estimation for discrete exponential-polynomial models which generally are non-normalized.

In this paper, we analyze classical variants of the Spectral Clustering (SC) algorithm in the Dynamic Stochastic Block Model (DSBM). Existing results show that, in the relatively sparse case where the expected degree grows logarithmically with the number of nodes, guarantees in the static case can be extended to the dynamic case and yield improved error bounds when the DSBM is sufficiently smooth in time, that is, the communities do not change too much between two time steps. We improve over these results by drawing a new link between the sparsity and the smoothness of the DSBM: the smoother the DSBM is, the more sparse it can be, while still guaranteeing consistent recovery. In particular, a mild condition on the smoothness allows to treat the sparse case with bounded degree. These guarantees are valid for the SC applied to the adjacency matrix or the normalized Laplacian. As a by-product of our analysis, we obtain to our knowledge the best spectral concentration bound available for the normalized Laplacian of matrices with independent Bernoulli entries.

In a binary classification problem where the goal is to fit an accurate predictor, the presence of corrupted labels in the training data set may create an additional challenge. However, in settings where likelihood maximization is poorly behaved—for example, if positive and negative labels are perfectly separable—then a small fraction of corrupted labels can improve performance by ensuring robustness. In this work, we establish that in such settings, corruption acts as a form of regularization, and we compute precise upper bounds on estimation error in the presence of corruptions. Our results suggest that the presence of corrupted data points is beneficial only up to a small fraction of the total sample, scaling with the square root of the sample size.

Statistical inference for discrete-valued time series has not been developed like traditional methods for time series generated by continuous random variables. Some relevant models exist, but the lack of a homogenous framework raises some critical issues. For instance, it is not trivial to explore whether models are nested and it is quite arduous to derive stochastic properties which simultaneously hold across different specifications. In this paper, inference for a general class of first order observation-driven models for discrete-valued processes is developed. Stochastic properties such as stationarity and ergodicity are derived under easy-to-check conditions, which can be directly applied to all the models encompassed in the class and for every distribution which satisfies mild moment conditions. Consistency and asymptotic normality of quasi-maximum likelihood estimators are established, with the focus on the exponential family. Finite sample properties and the use of information criteria for model selection are investigated throughout Monte Carlo studies. An empirical application to count data is discussed, concerning a test-bed time series on the spread of an infection.

We present a general methodology for using unlabeled data to design semi supervised learning (SSL) variants of the Empirical Risk Minimization (ERM) learning process. Focusing on generalized linear regression, we analyze of the effectiveness of our SSL approach in improving prediction performance. The key ideas are carefully considering the null model as a competitor, and utilizing the unlabeled data to determine signal-noise combinations where SSL outperforms both supervised learning and the null model. We then use SSL in an adaptive manner based on estimation of the signal and noise.

In the special case of linear regression with Gaussian covariates, we prove that the non-adaptive SSL version is in fact not capable of improving on both the supervised estimator and the null model simultaneously, beyond a negligible $O(1∕n)$ term. On the other hand, the adaptive model presented in this work, can achieve a substantial improvement over both competitors simultaneously, under a variety of settings. This is shown empirically through extensive simulations, and extended to other scenarios, such as non-Gaussian covariates, misspecified linear regression, or generalized linear regression with non-linear link functions.

We prove minimax bounds for estimating Gaussian location mixtures on ${\mathbb{R}}^d$ under the squared $L^2$ and the squared Hellinger loss functions. Under the squared $L^2$ loss, we prove that the minimax optimal rate is upper and lower bounded by a constant multiple of $n^{-1}{(logn)}^{d∕2}$. Under the squared Hellinger loss, we consider two subclasses based on the behavior of the tails of the mixing measure. When the mixing measure has a sub-Gaussian tail, the minimax rate under the squared Hellinger loss is bounded from below by ${(logn)}^d∕n$, which implies that the optimal minimax rate is between ${(logn)}^d∕n$ and the upper bound ${(logn)}^{d+1}∕n$ obtained by [11]. On the other hand, when the mixing measure is only assumed to have a bounded $p^{\text{th}}$ moment for a fixed $p>0$, the minimax rate under the squared Hellinger loss is bounded from below by $n^{-p∕(p+d)}{(logn)}^{-3d∕2}$. This rate is minimax optimal up to logarithmic factors.

Combining information both within and across trajectories, we propose a simple estimator for the local regularity of the trajectories of a stochastic process. Independent trajectories are measured with errors at randomly sampled time points. The proposed approach is model-free and applies to a large class of stochastic processes. Non-asymptotic bounds for the concentration of the estimator are derived. Given the estimate of the local regularity, we build a nearly optimal local polynomial smoother from the curves from a new, possibly very large sample of noisy trajectories. We derive non-asymptotic pointwise risk bounds uniformly over the new set of curves. Our estimates perform well in simulations, in both cases of differentiable or non-differentiable trajectories. Real data sets illustrate the effectiveness of the new approaches.

This paper deals with the drift estimation in linear stochastic evolution equations (with emphasis on linear SPDEs) with additive fractional noise (with Hurst index ranging from 0 to 1) via least-squares procedure. Since the least-squares estimator contains stochastic integrals of divergence type, we address the problem of its pathwise (and robust to observation errors) evaluation by comparison with the pathwise integral of Stratonovich type and using its chain-rule property. The resulting pathwise LSE is then defined implicitly as a solution to a non-linear equation. We study its numerical properties (existence and uniqueness of the solution) as well as statistical properties (strong consistency and the speed of its convergence). The asymptotic properties are obtained assuming fixed time horizon and increasing number of the observed Fourier modes (space asymptotics). We also conjecture the asymptotic normality of the pathwise LSE.

Ensemble Conditional Variance Estimation (ECVE) is a novel sufficient dimension reduction (SDR) method in regressions with continuous response and predictors. ECVE applies to general non-additive error regression models and operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. It is a semiparametric forward regression model-based exhaustive sufficient dimension reduction estimation method that is shown to be consistent under mild assumptions. ECVE outperforms central subspace mean average variance estimation (csMAVE), its main competitor, under several simulation settings and in a benchmark data set analysis.

We introduce the Generalized Rescaled Pólya (GRP) urn, that provides a generative model for a chi-squared test of goodness of fit for the long-term probabilities of clustered data, with independence between clusters and correlation, due to a reinforcement mechanism, inside each cluster. We apply the proposed test to a data set of Twitter posts about COVID-19 pandemic: in a few words, for a classical chi-squared test the data result strongly significant for the rejection of the null hypothesis (the daily long-run sentiment rate remains constant), but, taking into account the correlation among data, the introduced test leads to a different conclusion. Beside the statistical application, we point out that the GRP urn is a simple variant of the standard Eggenberger-Pólya urn, that, with suitable choices of the parameters, shows “local” reinforcement, almost sure convergence of the empirical mean to a deterministic limit and different asymptotic behaviours of the predictive mean. Moreover, the study of this model provides the opportunity to analyze stochastic approximation dynamics, that are unusual in the related literature.

Concentration inequalities are widely used for analyzing machine learning algorithms. However, the current concentration inequalities cannot be applied to some non-causal processes which appear for instance in Natural Language Processing (NLP). This is mainly due to the non-causal nature of such involved data, in the sense that each data point depends on other neighboring data points. In this paper, we establish a framework for modeling non-causal random fields and prove a Hoeffding-type concentration inequality. The proof of this result is based on a local approximation of the non-causal random field by a function of a finite number of i.i.d. random variables.

Factor model is an appealing and effective analytic tool for high-dimensional time series, with a wide range of applications in economics, finance and statistics. This paper develops two criteria for the determination of the number of factors for tensor factor models where the signal part of an observed tensor time series assumes a Tucker decomposition with the core tensor as the factor tensor. The task is to determine the dimensions of the core tensor. One of the proposed criteria is similar to information based criteria of model selection, and the other is an extension of the approaches based on the ratios of consecutive eigenvalues often used in factor analysis for panel time series. Theoretically results, including sufficient conditions and convergence rates, are established. The results include the vector factor models as special cases, with an additional convergence rates. Simulation studies provide promising finite sample performance for the two criteria.

We develop in this work a new dimension reduction method for high-dimensional settings. The proposed procedure is based on a principal support vector machine framework where principal projections are used in order to overcome the non-invertibility of the covariance matrix. Using a series of equivalences we show that one can accurately recover the central subspace using a projection on a lower dimensional subspace and then applying an ${\ell }_1$ penalization strategy to obtain sparse estimators of the sufficient directions. Based next on a desparsified estimator, we provide an inferential procedure for high-dimensional models that allows testing for the importance of variables in determining the sufficient direction. Theoretical properties of the methodology are illustrated and computational advantages are demonstrated with simulated and real data experiments.

The presence of measurement errors is a ubiquitously faced problem and plenty of work has been done to overcome this when a single covariate is mismeasured under a variety of conditions. However, in practice, it is possible that more than one covariate is measured with error. When measurements are taken by the same device, the errors of these measurements are likely correlated.

In this paper, we present a novel approach to estimate the covariance matrix of classical additive errors in the absence of validation data or auxiliary variables when two covariates are subject to measurement error. Our method assumes these errors to be following a bivariate normal distribution. We show that the variance matrix is identifiable under certain conditions on the support of the error-free variables and propose an estimation method based on an expansion of Bernstein polynomials. To investigate the performance of the proposed estimation method, the asymptotic properties of the estimator are examined and a diverse set of simulation studies is conducted. The estimated matrix is then used by the simulation-extrapolation (SIMEX) algorithm to reduce the bias caused by measurement error in logistic regression models. Finally, the method is demonstrated using data from the Framingham Heart Study.

In this paper we are concerned with a sequence of univariate random variables with piecewise polynomial means and independent sub-Gaussian noise. The underlying polynomials are allowed to be of arbitrary but fixed degrees. All the other model parameters are allowed to vary depending on the sample size.

We propose a two-step estimation procedure based on the ${\ell }_0$-penalisation and provide upper bounds on the localisation error. We complement these results by deriving global information-theoretic lower bounds, which show that our two-step estimators are nearly minimax rate-optimal. We also show that our estimator enjoys near optimally adaptive performance by attaining individual localisation errors depending on the level of smoothness at individual change points of the underlying signal. In addition, under a special smoothness constraint, we provide a minimax lower bound on the localisation errors. This lower bound is independent of the polynomial orders and is sharper than the global minimax lower bound.

Thanks to their simplicity and interpretable structure, auto-regressive processes are widely used to model time series data. However, many real time series data sets exhibit non-linear patterns, requiring non-linear modeling. The threshold Auto-Regressive (TAR) process provides a family of non-linear auto-regressive time series models in which the process dynamics are specific step functions of a thresholding variable. While estimation and inference for low-dimensional TAR models have been investigated, high-dimensional TAR models have received less attention. In this article, we develop a new framework for estimating high-dimensional TAR models, and propose two different sparsity-inducing penalties. The first penalty corresponds to a natural extension of classical TAR model to high-dimensional settings, where the same threshold is enforced for all model parameters. Our second penalty develops a more flexible TAR model, where different thresholds are allowed for different auto-regressive coefficients. We show that both penalized estimation strategies can be utilized in a three-step procedure that consistently learns both the thresholds and the corresponding auto-regressive coefficients. However, our theoretical and empirical investigations show that the direct extension of the TAR model is not appropriate for high-dimensional settings and is better suited for moderate dimensions. In contrast, the more flexible extension of the TAR model leads to consistent estimation and superior empirical performance in high dimensions.

We revisit the problem of designing an efficient binary classifier in a challenging high-dimensional framework. The model under study assumes some local dependence structure among feature variables represented by a block-diagonal covariance matrix with a growing number of blocks of an arbitrary, but fixed size. The blocks correspond to non-overlapping independent groups of strongly correlated features. To assess the relevance of a particular block in predicting the response, we introduce a measure of “signal strength” pertaining to each feature block. This measure is then used to specify a sparse model of our interest. We further propose a threshold-based feature selector which operates as a screen-and-clean scheme integrated into a linear classifier: the data is subject to screening and hard threshold cleaning to filter out the blocks that contain no signals. Asymptotic properties of the proposed classifiers are studied when the sample size n depends on the number of feature blocks b, and the sample size goes to infinity with b at a slower rate than b. The new classifiers, which are fully adaptive to unknown parameters of the model, are shown to perform asymptotically optimally in a large part of the classification region. The numerical study confirms good analytical properties of the new classifiers that compare favorably to the existing threshold-based procedure used in a similar context.

Sufficient Dimension Reduction (SDR) becomes an important tool for mitigating the curse of dimensionality in high dimensional regression analysis. Recently, Flexible SDR (FSDR) has been proposed to extend SDR by finding lower dimensional projections of transformed explanatory variables. The dimensions of the projections however cannot fully represent the extent of data reduction FSDR can achieve. As a consequence, optimality and other theoretical properties of FSDR are currently not well understood. In this article, we propose to use the σ-field associated with the projections, together with their dimensions to fully characterize FSDR, and refer to the σ-field as the FSDR σ-field. We further introduce the concept of minimal FSDR σ-field and consider FSDR projections with the minimal σ-field optimal. Under some mild conditions, we show that the minimal FSDR σ-field exists, attaining the lowest dimensionality at the same time. To estimate the minimal FSDR σ-field, we propose a two-stage procedure called the Generalized Kernel Dimension Reduction (GKDR) method and partially establish its consistency property under weak conditions. Extensive simulation experiments demonstrate that the GKDR method can effectively find the minimal FSDR σ-field and outperform other existing methods. The application of GKDR to a real life air pollution data set sheds new light on the connections between atmospheric conditions and air quality.

In the sparse sequence model, we consider a popular Bayesian multiple testing procedure and investigate for the first time its behaviour from the frequentist point of view. Given a spike-and-slab prior on the high-dimensional sparse unknown parameter, one can easily compute posterior probabilities of coming from the spike, which correspond to the well known local-fdr values [27], also called ℓ-values. The spike-and-slab weight parameter is calibrated in an empirical Bayes fashion, using marginal maximum likelihood. The multiple testing procedure under study, called here the cumulative ℓ-value procedure, ranks coordinates according to their empirical ℓ-values and thresholds so that the cumulative ranked sum does not exceed a user-specified level t. We validate the use of this method from the multiple testing perspective: for alternatives of appropriately large signal strength, the false discovery rate (FDR) of the procedure is shown to converge to the target level t, while its false negative rate (FNR) goes to 0. We complement this study by providing convergence rates for the method. Additionally, we prove that the q-value multiple testing procedure [44, 17] shares similar convergence rates in this model.

Sufficient dimension reduction (SDR) is an important tool in regression analysis which reduces the dimension of covariates without losing predictive information. Several methods have been proposed to handle data with either censoring in the response or measurement error in covariates. However, little research is available to deal with data having these two features simultaneously. In this paper, we examine this problem. We start with considering the cumulative distribution function in regular settings and propose a valid SDR method to incorporate the effects of censored data and covariates measurement error. Theoretical results are established, and numerical studies are reported to assess the performance of the proposed methods.

We propose a nonparametric estimator of the expected discounted penalty function in the compound Poisson risk model. We use a projection estimator on the Laguerre basis and we compute the coefficients using Plancherel theorem. We provide an upper bound on the MISE of our estimator, and we show it achieves parametric rates of convergence on Sobolev–Laguerre spaces without needing a bias-variance compromise. Moreover, we compare our estimator with the Laguerre deconvolution method. We compute an upper bound of the MISE of the Laguerre deconvolution estimator and we compare it on Sobolev–Laguerre spaces with our estimator. Finally, we compare these estimators on simulated data.

Dependency structure in recommender systems has been widely adopted in recent years to improve prediction accuracy. In this paper, we propose an innovative tensor-based recommender system, namely, the Tensor Factorization with Dependency (TFD). The proposed method utilizes shared factors to characterize the dependency between different modes, in addition to pairwise additive tensor factorization to integrate information among multiple modes. One advantage of the proposed method is that it provides flexibility for different dependency structures by incorporating shared latent factors. In addition, the proposed method unifies both binary and ordinal ratings in recommender systems. We achieve scalable computation for scarce tensors with high missing rates. In theory, we show the asymptotic consistency of estimators with various loss functions for both binary and ordinal data. Our numerical studies demonstrate that the proposed method outperforms the existing methods, especially on prediction accuracy.

Working with so-called linkages allows to define a copula-based, $[0,1]$-valued multivariate dependence measure ${\mathit{\zeta }}^1(X,Y)$ quantifying the scale-invariant extent of dependence of a random variable Y on a d-dimensional random vector $X=(X_1,\dots ,X_d)$ which exhibits various good and natural properties. In particular, ${\mathit{\zeta }}^1(X,Y)=0$ if and only if X and Y are independent, ${\mathit{\zeta }}^1(X,Y)$ is maximal exclusively if Y is a function of X, and ignoring one or several coordinates of X can not increase the resulting dependence value. After introducing and analyzing the metric $D_1$ underlying the construction of the dependence measure and deriving examples showing how much information can be lost by only considering all pairwise dependence values ${\mathit{\zeta }}^1(X_1,Y),\dots ,{\mathit{\zeta }}^1(X_d,Y)$ we derive a so-called checkerboard estimator for ${\mathit{\zeta }}^1(X,Y)$ and show that it is strongly consistent in full generality, i.e., without any smoothness restrictions on the underlying copula. Some simulations illustrating the small sample performance of the estimator complement the established theoretical results.

Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance—an easily computable variant of the Wasserstein distance. Specifically, we construct confidence intervals for the Sliced Wasserstein distance which have finite-sample validity under no assumptions or under mild moment assumptions. These intervals are adaptive in length to the regularity of the underlying distributions. We also bound the minimax risk of estimating the Sliced Wasserstein distance, and as a consequence establish that the lengths of our proposed confidence intervals are minimax optimal over appropriate distribution classes. To motivate the choice of these classes, we also study minimax rates of estimating a distribution under the Sliced Wasserstein distance. These theoretical findings are complemented with a simulation study demonstrating the deficiencies of the classical bootstrap, and the advantages of our proposed methods. We also show strong correspondences between our theoretical predictions and the adaptivity of our confidence interval lengths in simulations. We conclude by demonstrating the use of our confidence intervals in the setting of simulator-based likelihood-free inference. In this setting, contrasting popular approximate Bayesian computation methods, we develop uncertainty quantification methods with rigorous frequentist coverage guarantees.

We discuss Bayesian nonparametric procedures for the regression analysis of compositional responses, that is, data supported on a multivariate simplex. The procedures are based on a modified class of multivariate Bernstein polynomials and on the use of dependent stick-breaking processes. A general model and two simplified versions of the general model are discussed. Appealing theoretical properties such as continuity, association structure, support, and consistency of the posterior distribution are established. Additionally, we exploit the use of spike-and-slab priors for choosing the version of the model that best adapts to the complexity of the underlying true data-generating distribution. The performance of the proposed model is illustrated in a simulation study and in an application to solid waste data from Colombia.

Exponential tail bounds for sums play an important role in statistics, but the example of the t-statistic shows that the exponential tail decay may be lost when population parameters need to be estimated from the data. However, it turns out that if Studentizing is accompanied by estimating the location parameter in a suitable way, then the t-statistic regains the exponential tail behavior. Motivated by this example, the paper analyzes other ways of empirically standardizing sums and establishes tail bounds that are sub-Gaussian or even closer to normal for the following settings: Standardization with Studentized contrasts for normal observations, standardization with the log likelihood ratio statistic for observations from an exponential family, and standardization via self-normalization for observations from a symmetric distribution with unknown center of symmetry. The latter standardization gives rise to a novel scan statistic for heteroscedastic data whose asymptotic power is analyzed in the case where the observations have a log-concave distribution.

We study inference for the driving Lévy noise of an ergodic stochastic differential equation (SDE) model, when the process is observed at high-frequency and long time and when the drift and scale coefficients contain finite-dimensional unknown parameters. By making use of the Gaussian quasi-likelihood function for the coefficients, we derive a stochastic expansion for functionals of the unit-time residuals, which clarifies some quantitative effect of plugging in the estimators of the coefficients, thereby enabling us to take several inference procedures for the driving-noise characteristics into account. We also present new classes and methods available in YUIMA for the simulation and the estimation of a Lévy SDE model. We highlight the flexibility of these new advances in YUIMA using simulated and real data.

A representative model in integrative analysis of two high-dimensional correlated datasets is to decompose each data matrix into a low-rank common matrix generated by latent factors shared across datasets, a low-rank distinctive matrix corresponding to each dataset, and an additive noise matrix. Existing decomposition methods claim that their common matrices capture the common pattern of the two datasets. However, their so-called common pattern only denotes the common latent factors but ignores the common pattern between the two coefficient matrices of these common latent factors. We propose a new unsupervised learning method, called the common and distinctive pattern analysis (CDPA), which appropriately defines the two types of data patterns by further incorporating the common and distinctive patterns of the coefficient matrices. A consistent estimation approach is developed for high-dimensional settings, and shows reasonably good finite-sample performance in simulations. Our simulation studies and real data analysis corroborate that the proposed CDPA can provide better characterization of common and distinctive patterns and thereby benefit data mining.

We propose a Bayesian semiparametric approach for registration of multiple point processes. Our approach entails modelling the mean measures of the phase-varying point processes with a Bernstein–Dirichlet prior, which induces a prior on the space of all warp functions. Theoretical results on the support of the induced priors are derived, and posterior consistency is obtained under mild conditions. Numerical experiments suggest a good performance of the proposed methods, and a climatology real-data example is used to showcase how the method can be employed in practice.

The smooth bootstrap for estimating copula functionals in small samples is investigated. It can be used both to gauge the distribution of the estimator in question and to augment the data. Issues arising from kernel density and distribution estimation in the copula domain are addressed, such as how to avoid the bounded domain, which bandwidth matrix to choose, and how the smoothing can be carried out. Furthermore, we investigate how the smooth bootstrap impacts the underlying dependence structure or the functionals in question and under which conditions it does not. We provide specific examples and simulations that highlight advantages and caveats of the approach.

Staged tree models are a discrete generalization of Bayesian networks. We show that these form curved exponential families and derive their natural parameters, sufficient statistic, and cumulant-generating function as functions of their graphical representation. We give necessary and sufficient graphical criteria for classifying regular subfamilies and discuss implications for model selection.

Functional data analysis (FDA) methods have computational and theoretical appeals for some high dimensional data, but lack the scalability to modern large sample datasets. To tackle the challenge, we develop randomized algorithms for two important FDA methods: functional principal component analysis (FPCA) and functional linear regression (FLR) with scalar response. The two methods are connected as they both rely on the accurate estimation of functional principal subspace. The proposed algorithms draw subsamples from the large dataset at hand and apply FPCA or FLR over the subsamples to reduce the computational cost. To effectively preserve subspace information in the subsamples, we propose a functional principal subspace sampling probability, which removes the eigenvalue scale effect inside the functional principal subspace and properly weights the residual. Based on the operator perturbation analysis, we show the proposed probability has precise control over the first order error of the subspace projection operator and can be interpreted as an importance sampling for functional subspace estimation. Moreover, concentration bounds for the proposed algorithms are established to reflect the low intrinsic dimension nature of functional data in an infinite dimensional space. The effectiveness of the proposed algorithms is demonstrated upon synthetic and real datasets.

Tensor regression models are of emerging interest in diverse fields of social and behavioral sciences, including neuroimaging analysis, neural networks, image processing and so on. Recent theoretical advancements of tensor decomposition have facilitated significant development of various tensor regression models. The focus of most of the available literature has been on the Canonical Polyadic (CP) decomposition and its variants for the regression coefficient tensor. A CP decomposed coefficient tensor enables estimation with relatively small sample size, but it may not always capture the underlying complex structure in the data. In this work, we leverage the recently developed concept of tubal rank and develop a tensor regression model, wherein the coefficient tensor is decomposed into two components: a low tubal rank tensor and a structured sparse one. We first address the issue of identifiability of the two components comprising the coefficient tensor and subsequently develop a fast and scalable Alternating Minimization algorithm to solve the convex regularized program. Further, we provide finite sample error bounds under high dimensional scaling for the model parameters. The performance of the model is assessed on synthetic data and is also used in an application involving data from an intelligent tutoring platform.

For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.

We consider the estimation of the global mode of a density under some decay rate condition around the global mode. We show that the maximum of a histogram, with proper choice of bandwidth, achieves the minimax rate that we establish for the setting that we consider. This is based on knowledge of the decay rate. To address the situation where the decay rate is unknown, we propose a multiscale variant consisting in the recursive refinement of a histogram. We show that this variant is minimax adaptive. These methods run in linear time, and we prove in an appendix that this is best possible: There is no estimation procedure running in sublinear time that achieves the minimax rate.

In this paper, we explore the multiple testing problem of paired null hypotheses, for which the data are collected on pairs of entities and tests have to be performed for each pair. Typically, for each pair $(i,j)$, we observe some interaction/association score between i and j and the aim is to detect the pairs with a significant score. In this setting, it is natural to assume that the true/false null constellation is structured according to an unobserved graph, where present edges correspond to a significant association score. The point of this work is to build an improved multiple testing decision by learning the graph structure. Our approach is in line with the seminal work of Sun and Cai [46], that uses the hidden Markov model to structure the dependencies between null hypotheses. Here, we adapt this strategy by considering the stochastic block model for the latent graph. Under appropriate assumptions, the new proposed procedure is shown to control the false discovery rate, up to remainder terms that vanish when the size of the number of hypotheses increases. The procedure is also shown to be nearly optimal in the sense that it is close to the procedure maximizing the true discovery rate. Numerical experiments reveal that our method outperforms state-of-the-art methods and is robust to model mis-specification. Finally, the applicability of the new method is demonstrated on data concerning the usage of self-service bicycles in London.

The $L^2$-approximation of occupation and local times of a symmetric α-stable Lévy process from high frequency discrete time observations is studied. The standard Riemann sum estimators are shown to be asymptotically efficient when $0<\mathit{\alpha }\le 1$, but only rate optimal for $1<\mathit{\alpha }\le 2$. For this, the exact convergence of the $L^2$-approximation error is proven with explicit constants.

We consider truncated (or conditional) sum-of-squares estimation of a parametric fractional time series model with an additive deterministic structure. The latter consists of both a drift term and a generalized power law trend. The memory parameter of the stochastic component and the power parameter of the deterministic trend component are both considered unknown real numbers to be estimated and belonging to arbitrarily large compact sets. Thus, our model captures different forms of nonstationarity and noninvertibility as well as a very flexible deterministic specification. As in related settings, the proof of consistency (which is a prerequisite for proving asymptotic normality) is challenging due to non-uniform convergence of the objective function over a large admissible parameter space and due to the competition between stochastic and deterministic components. As expected, parameter estimates related to the deterministic component are shown to be consistent and asymptotically normal only for parts of the parameter space depending on the relative strength of the stochastic and deterministic components. In contrast, we establish consistency and asymptotic normality of parameter estimates related to the stochastic component for the entire parameter space. Furthermore, the asymptotic distribution of the latter estimates is unaffected by the presence of the deterministic component, even when this is not consistently estimable. We also include Monte Carlo simulations to illustrate our results.

We explicitly quantify the empirically observed phenomenon that estimation under a stochastic block model (SBM) is hard if the model contains classes that are similar. More precisely, we consider estimation of certain functionals of random graphs generated by a SBM. The SBM may or may not be sparse, and the number of classes may be fixed or grow with the number of vertices. Minimax lower and upper bounds of estimation along specific submodels are derived. The results are nonasymptotic and imply that uniform estimation of a single connectivity parameter is much slower than the expected asymptotic pointwise rate. Specifically, the uniform quadratic rate does not scale as the number of edges, but only as the number of vertices. The lower bounds are local around any possible SBM. An analogous result is derived for functionals of a class of smooth graphons.

Initially designed for independent datas, low-rank matrix completion was successfully applied in many domains to the reconstruction of partially observed high-dimensional time series. However, there is a lack of theory to support the application of these methods to dependent datas. In this paper, we propose a general model for multivariate, partially observed time series. We show that the least-square method with a rank penalty leads to reconstruction error of the same order as for independent datas. Moreover, when the time series has some additional properties such as periodicity or smoothness, the rate can actually be faster than in the independent case.

Suppose that a statistician observes two independent variates $X_1$ and $X_2$ having densities $f_i(\cdot ;\mathit{\theta })\equiv f_i(\cdot -\mathit{\theta }),i=1,2$, $\mathit{\theta }\in \mathbb{R}$. His purpose is to conduct a test for

$$H:\mathit{\theta }=0\text{vs.}K:\mathit{\theta }\in \mathbb{R}\setminus \{0\}$$

with a pre-defined significance level $\mathit{\alpha }\in (0,1)$. Moran (1973) suggested a test which is based on a single split of the data, i.e., to use $X_2$ in order to conduct a one-sided test in the direction of $X_1$. Specifically, if $b_1$ and $b_2$ are the $(1-\mathit{\alpha })$’th and α’th quantiles associated with the distribution of $X_2$ under H, then Moran’s test has a rejection zone

$$(a,\mathrm{\infty })\times (b_1,\mathrm{\infty })\cup (-\mathrm{\infty },a)\times (-\mathrm{\infty },b_2)$$

where $a\in \mathbb{R}$ is a design parameter. Motivated by this issue, the current work includes an analysis of a new notion, regular admissibility of tests. It turns out that the theory regarding this kind of admissibility leads to a simple sufficient condition on $f_1(\cdot )$ and $f_2(\cdot )$ under which Moran’s test is inadmissible.

We establish a functional central limit theorem for the empirical Ripley’s K-function of Gibbs point processes and point processes with fast decay of correlations. Our theorem greatly extend past results that were restricted to the Poisson case and allow to determine the asymptotic behaviour of statistics based on the K-function which may be used, for example, to develop goodness-of-fit tests. We illustrate this in a simulation study.

Deriving the limiting distribution of a nonparametric estimate is rather challenging but of fundamental importance to statistical inference. For the current status data, we study a penalized nonparametric likelihood-based estimator for an unknown cumulative hazard function, and establish the pointwise asymptotic normality of the resulting nonparametric estimate. We also propose the penalized likelihood ratio tests for local and global hypotheses, derive their limiting distributions, and study the optimality of the global test. Simulation studies show that the proposed method works well compared to the classical likelihood ratio test.

The stochastic blockmodel (SBM) models the connectivity within and between disjoint subsets of nodes in networks. Prior work demonstrated that the rows of an SBM’s adjacency spectral embedding (ASE) and Laplacian spectral embedding (LSE) both converge in law to Gaussian mixtures where the components are curved exponential families. Maximum likelihood estimation via the Expectation-Maximization (EM) algorithm for a full Gaussian mixture model (GMM) can then perform the task of clustering graph nodes, albeit without appealing to the components’ curvature. Noting that EM is a special case of the Expectation-Solution (ES) algorithm, we propose two ES algorithms that allow us to take full advantage of these curved structures. After presenting the ES algorithm for the general curved-Gaussian mixture, we develop those corresponding to the ASE and LSE limiting distributions. Simulating from artificial SBMs and a brain connectome SBM reveals that clustering graph nodes via our ES algorithms can improve upon that of EM for a full GMM for a wide range of settings.

In this paper, we study the distribution of the so-called “Yule’s nonsense correlation statistic” on a time interval $[0,T]$ for a time horizon $T>0$, when T is large, for a pair $(X_1,X_2)$ of independent Ornstein-Uhlenbeck processes. This statistic is by definition equal to:

$$\mathit{\rho }(T):=\frac{Y_{12}(T)}{\sqrt{Y_{11}(T)}\sqrt{Y_{22}(T)}},$$

where the random variables $Y_{ij}(T)$, $i,j=1,2$ are defined as

$$Y_{ij}(T):={\int }_0^T{X}_i(u)X_j(u)du-T{\overline{X}}_i\overline{X_j},{\overline{X}}_i:=\frac{1}{T}{\int }_0^T{X}_i(u)du.$$

We assume $X_1$ and $X_2$ have the same drift parameter $\mathit{\theta }>0$. We also study the asymptotic law of a discrete-type version of $\mathit{\rho }(T)$, where $Y_{ij}(T)$ above are replaced by their Riemann-sum discretizations. In this case, conditions are provided for how the discretization (in-fill) step relates to the long horizon T. We establish identical normal asymptotics for standardized $\mathit{\rho }(T)$ and its discrete-data version. The asymptotic variance of $\mathit{\rho }(T)T^{1∕2}$ is ${\mathit{\theta }}^{-1}$. We also establish speeds of convergence in the Kolmogorov distance, which are of Berry-Esséen-type (constant*$T^{-1∕2}$) except for a $lnT$ factor. Our method is to use the properties of Wiener-chaos variables, since $\mathit{\rho }(T)$ and its discrete version are comprised of ratios involving three such variables in the 2nd Wiener chaos. This methodology accesses the Kolmogorov distance thanks to a relation which stems from the connection between the Malliavin calculus and Stein’s method on Wiener space.

In this paper, we consider a one-dimensional diffusion process with jumps driven by a Hawkes process. We are interested in the estimations of the volatility function and of the jump function from discrete high-frequency observations in a long time horizon which remained an open question until now. First, we propose to estimate the volatility coefficient. For that, we introduce a truncation function in our estimation procedure that allows us to take into account the jumps of the process and estimate the volatility function on a linear subspace of $L^2(A)$ where A is a compact interval of $\mathbb{R}$. We obtain a bound for the empirical risk of the volatility estimator, ensuring its consistency, and then we study an adaptive estimator w.r.t. the regularity. Then, we define an estimator of a sum between the volatility and the jump coefficient modified with the conditional expectation of the intensity of the jumps. We also establish a bound for the empirical risk for the non-adaptive estimators of this sum, the convergence rate up to the regularity of the true function, and an oracle inequality for the final adaptive estimator.

Finally, we give a methodology to recover the jump function in some applications. We conduct a simulation study to measure our estimators’ accuracy in practice and discuss the possibility of recovering the jump function from our estimation procedure.

Considering two random variables with different laws to which we only have access through finite size i.i.d samples, we address how to reweight the first sample so that its empirical distribution converges towards the true law of the second sample as the size of both samples goes to infinity. We study an optimal reweighting that minimizes the Wasserstein distance between the empirical measures of the two samples, and leads to an expression of the weights in terms of Nearest Neighbors. The consistency and some asymptotic convergence rates in terms of expected Wasserstein distance are derived, and do not need the assumption of absolute continuity of one random variable with respect to the other. These results have some application in Uncertainty Quantification for decoupled estimation and in the bound of the generalization error for the Nearest Neighbor regression under covariate shift.

Community detection is a fundamental problem in network analysis with many methods available to estimate communities. Most of these methods assume that the number of communities is known, which is often not the case in practice. We study a simple and very fast method for estimating the number of communities based on the spectral properties of certain graph operators, such as the non-backtracking matrix and the Bethe Hessian matrix. We show that the method performs well under several models and a wide range of parameters, and is guaranteed to be consistent under several asymptotic regimes. We compare this method to several existing methods for estimating the number of communities and show that it is both more accurate and more computationally efficient.

The nearest shrunken centroids classifier (NSC) is a popular high-dimensional classifier. However, it is prone to inaccurate classification when the data is heavy-tailed. In this paper, we develop a robust generalization of NSC (RNSC) which remains effective under such circumstances. By incorporating the Huber loss both in the estimation and the calculation of the score function, we reduce the impacts of heavy tails. We rigorously show the variable selection, estimation, and prediction consistency in high dimensions under weak moment conditions. Empirically, our proposal greatly outperforms NSC and many other successful classifiers when data is heavy-tailed while remaining comparable to NSC in the absence of heavy tails. The favorable performance of RNSC is also demonstrated in a real data example.

We provide an empirical process theory for locally stationary processes over nonsmooth function classes. An important novelty over other approaches is the use of the flexible functional dependence measure to quantify dependence. A functional central limit theorem and nonasymptotic maximal inequalities are provided. The theory is used to prove the functional convergence of the empirical distribution function (EDF) and to derive uniform convergence rates for kernel density estimators both for stationary and locally stationary processes. A comparison with earlier results based on other measures of dependence is carried out.

We establish statistical properties of random-weighting methods in LASSO regression under different regularization parameters ${\mathit{\lambda }}_n$ and suitable regularity conditions. The random-weighting methods in view concern repeated optimization of a randomized objective function, motivated by the need for computationally efficient uncertainty quantification in contemporary estimation settings. In the context of LASSO regression, we repeatedly assign analyst-drawn random weights to terms in the objective function, and optimize to obtain a sample of random-weighting estimators. We show that existing approaches have conditional model selection consistency and conditional asymptotic normality at different growth rates of ${\mathit{\lambda }}_n$ as $n\to \mathrm{\infty }$. We propose an extension to the available random-weighting methods and establish that the resulting samples attain conditional sparse normality and conditional consistency in a growing-dimension setting. We illustrate the proposed methodology using synthetic and benchmark data sets, and we discuss the relationship of the results to approximate nonparametric Bayesian analysis and to perturbation bootstrap methods.

We consider $X_1,\dots ,X_n$ a sample of data on the circle ${\mathbb{S}}^1$, whose distribution is a two-component mixture. Denoting R and Q two rotations on ${\mathbb{S}}^1$, the density of the $X_i$’s is assumed to be $g(x)=pf(R^{-1}x)+(1-p)f(Q^{-1}x)$, where $p\in (0,1)$ and f is an unknown density on the circle. In this paper we estimate both the parametric part $\mathit{\theta }=(p,R,Q)$ and the nonparametric part f. The specific problems of identifiability on the circle are studied. A consistent estimator of θ is introduced and its asymptotic normality is proved. We propose a Fourier-based estimator of f with a penalized criterion to choose the resolution level. We show that our adaptive estimator is optimal from the oracle and minimax points of view when the density belongs to a Sobolev ball. Our method is illustrated by numerical simulations.

Inferring the heterogeneous treatment effect is a fundamental problem in many applications. In this paper, we focus on estimating the Conditional Average Treatment Effect (CATE), that is, the difference in the conditional mean outcome between treatments given covariates. Traditionally, Q-Learning based approaches estimate each conditional mean outcome. However, they are subject to model misspecification. Recently, flexible one-step methods to directly learn (D-Learning) the CATE without outcome model specifications have been proposed. However, they require a specification of the propensity score. We propose robust direct learning (RD-Learning), to augment D-learning, leading to doubly robust estimators of the treatment effect. The consistency for our CATE estimator is guaranteed if either the main effect model or the propensity score model is correctly specified. The framework can be used in both the binary and the multi-arm settings and is general enough to allow different function spaces and incorporate different generic learning algorithms. We conduct a thorough theoretical analysis of the prediction error of our CATE estimator using statistical learning theory under both linear and non-linear settings. The effectiveness of our proposed method is demonstrated by simulation studies and a real data example about an AIDS Clinical Trials study.

Cluster indices describe extremal behaviour of stationary time series. We consider runs estimators of cluster indices. Using a modern theory of multivariate, regularly varying time series, we obtain central limit theorems under conditions that can be easily verified for a large class of models. In particular, we show that blocks and runs estimators have the same limiting variance.

Change point detection in high dimensional data has found considerable interest in recent years. Most of the literature either designs methodology for a retrospective analysis, where the whole sample is already available when the statistical inference begins, or considers online detection schemes controlling the average time until a false alarm. This paper takes a different point of view and develops monitoring schemes for the online scenario, where high dimensional data arrives successively and the goal is to detect changes as fast as possible controlling at the same time the probability of a type I error of a false alarm. We develop a sequential procedure capable of detecting changes in the mean vector of a successively observed high dimensional time series with spatial and temporal dependence. The statistical properties of the method are analyzed in the case where both, the sample size and dimension tend to infinity. In this scenario, it is shown that the new monitoring scheme has asymptotic level alpha under the null hypothesis of no change and is consistent under the alternative of a change in at least one component of the high dimensional mean vector. The approach is based on a new type of monitoring scheme for one-dimensional data which turns out to be often more powerful than the usually used CUSUM and Page-CUSUM methods, and the component-wise statistics are aggregated by the maximum statistic. For the analysis of the asymptotic properties of our monitoring scheme we prove that the range of a Brownian motion on a given interval is in the domain of attraction of the Gumbel distribution, which is a result of independent interest in extreme value theory. The finite sample properties of the new methodology are illustrated by means of a simulation study and in the analysis of a data example.

We define a moment-based estimator that maximizes the empirical saddlepoint (ESP) approximation of the distribution of solutions to empirical moment conditions. We call it the ESP estimator. We prove its existence, consistency and asymptotic normality, and we propose novel test statistics. We also show that the ESP estimator corresponds to the MM (method of moments) estimator shrunk toward parameter values with lower implied estimated variance, so it reduces the documented instability of existing moment-based estimators. In the case of just-identified moment conditions, which is the case we focus on, the ESP estimator is different from the MM estimator, unlike the more recent alternatives, such as the empirical-likelihood-type estimators.

We present a new finite-sample analysis of M-estimators of locations in a Hilbert space using the tool of the influence function. In particular, we show that the deviations of an M-estimator can be controlled thanks to its influence function (or its score function) and then, we use concentration inequality on M-estimators to investigate the robust estimation of the mean in high dimension in a corrupted setting (adversarial corruption setting) for bounded and unbounded score functions. For a sample of size n and covariance matrix Σ, we attain the minimax speed $\sqrt{Tr(\mathrm{\Sigma })∕n}+\sqrt{\Vert \mathrm{\Sigma }{\Vert }_{op}log(1∕\mathit{\delta })∕n}$ with probability larger than $1-\mathit{\delta }$ in a heavy-tailed setting. One of the major advantages of our approach compared to others recently proposed is that our estimator is tractable and fast to compute even in very high dimension with a complexity of $O(ndlog(Tr(\mathrm{\Sigma })))$ where n is the sample size and Σ is the covariance matrix of the inliers and in the code that we make available for this article is tested to be very fast.