We study the problem of estimating the surface area of the boundary $\partial S$ of a sufficiently smooth set $S\subset {\mathbb{R}}^d$ when the available information is only a finite subset ${\mathbb{X}}_n\subset S$. We propose two estimators. The first makes use of the Devroye–Wise support estimator and is based on Crofton’s formula, which, roughly speaking, states that the $(d-1)$-dimensional surface area of a smooth enough set is the mean number of intersections of randomly chosen lines. For that purpose, we propose an estimator of the number of intersections of such lines with support based on the Devroye–Wise support estimators. The second surface area estimator makes use of the α-convex hull of ${\mathbb{X}}_n$, which is denoted by $C_{\mathit{\alpha }}({\mathbb{X}}_n)$. More precisely, it is the $(d-1)$-dimensional surface area of $C_{\mathit{\alpha }}({\mathbb{X}}_n)$, as denoted by $|C_{\mathit{\alpha }}({\mathbb{X}}_n){|}_{d-1}$, which is proven to converge to the $(d-1)$-dimensional surface area of $\partial S$. Moreover, $|C_{\mathit{\alpha }}({\mathbb{X}}_n){|}_{d-1}$ can be computed using Crofton’s formula.

Our results depend on the Hausdorff distance between S and ${\mathbb{X}}_n$ for the Devroye–Wise estimator, and the Hausdorff distance between $\partial S$ and $\partial C_{\mathit{\alpha }}({\mathbb{X}}_n)$ for the second estimator.

In this paper, we propose the nonparametric empirical Bayes (NPEB) estimator based on the nonparametric maximum likelihood estimation (NPMLE) in Poisson mean vector estimation, also known as the g-modeling in the nonparametric empirical Bayes method. Due to the recent developments of highly scalable algorithms of empirical Bayes, it is more attractive to use g-modelling, while most of the studies have focused on the performance of f-modeling in the NPEB estimator. We study the theoretical properties of the NPEB estimator of Poisson mean vector based on g-modeling combined with the NPMLE, such as the convergence rate, and compare our result with some existing studies. Our simulation studies and real data examples of protein domain data show that the estimator based on the g-modeling outperforms existing f-modeling based estimators in both computational efficiency and accuracy.

We study the non-parametric isotonic regression problem for bivariate elicitable functionals that are given as an elicitable univariate functional and its Bayes risk. Prominent examples for functionals of this type are (mean, variance) and (Value-at-Risk, Expected Shortfall), where the latter pair consists of important risk measures in finance. We present our results for totally ordered covariates but extenstions to partial orders are given in the appendix.

Graphical models are useful to characterize the dependence structure of variables and have been commonly used for analysis of complex structured data. While various estimation methods have been developed under different graphical models, those methods are, however, inadequate to handle noisy data with measurement error. The development of most existing approaches relies on the implicit yet stringent assumption that the associated variables must be measured precisely. This assumption is unrealistic for many applications because mismeasurement in variables is usually presented in the data collection process. In this paper, we consider analysis of error-prone data under graphical models. To understand the impact of measurement error, we first study the asymptotic bias of the naive analysis which disregards the feature of measurement error in the variables. Furthermore, we develop a de-noising estimation procedure to account for measurement error effects. Theoretical results are established for the proposed method and numerical studies are reported to assess the finite sample performance of our proposed method.

Determining the number of clusters is crucial for the successful application of clustering. In this paper, we propose a new order-determination method called the data augmentation estimator (DAE), for the general model-based clustering. The estimator is based on a novel idea that augments data with an independently generated small cluster, which enables us to justify how the instability of clustering changes with the number of clusters assumed in clustering. The pattern of instability provides an alternative characterization of the true number of clusters to the commonly used goodness-of-fit measure. By combining the two sources of information appropriately, the proposed estimator reaches asymptotic consistency under general conditions and is easily implementable. It is also more efficient than the conventional BIC-type approaches that use the goodness-of-fit measure only. These properties are illustrated by the simulation studies and real data examples at the end.

This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in x for all values of z. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be Hölder smooth in x, but also be total variation smooth in z. We propose a corresponding kernel-based estimator and prove that it achieves the minimax rate. We give some simple examples of densities satisfying our assumptions which imply that our results are not vacuous. Finally, we propose an estimator which achieves the minimax optimal rate adaptively, i.e., without the need to know the smoothness parameter values in advance. Crucially, both of our estimators (the adaptive and non-adaptive ones) impose no assumptions on the marginal density $p_Z$, and are not obtained as a ratio between two kernel smoothing estimators which may sound like a go to approach in this problem.

In this study, we develop an asymptotic theory of nonparametric regression for a locally stationary functional time series. First, we introduce the notion of a locally stationary functional time series (LSFTS) that takes values in a semi-metric space. Then, we propose a nonparametric model for LSFTS with a regression function that changes smoothly over time. We establish the uniform convergence rates of a class of kernel estimators, the Nadaraya-Watson (NW) estimator of the regression function, and a central limit theorem of the NW estimator.

We propose a sieve M-estimation procedure which combines the flexibility of semiparametric inference with the stability and reliability of infinitesimal robustness. We derive the asymptotic theory of the proposed estimators, studying their convergence rate. In the context of functional magnetic resonance imaging (fMRI) data analysis, we illustrate how to apply our procedure to conduct inference on a semiparametric dynamic factor model. Monte Carlo simulations and real data analysis exemplify the stability of our estimators, providing a comparison with the extant, non robust and routinely applied sieve M-estimators.

Modern-day problems in statistics often face the challenge of exploring and analyzing complex non-Euclidean object data that do not conform to vector space structures or operations. Examples of such data objects include covariance matrices, graph Laplacians of networks, and univariate probability distribution functions. In the current contribution a new concurrent regression model is proposed to characterize the time-varying relation between an object in a general metric space (as a response) and a vector in ${\mathbb{R}}^p$ (as a predictor), where concepts from Fréchet regression is employed. Concurrent regression has been a well-developed area of research for Euclidean predictors and responses, with many important applications for longitudinal studies and functional data. However, there is no such model available so far for general object data as responses. We develop generalized versions of both global least squares regression and locally weighted least squares smoothing in the context of concurrent regression for responses which are situated in general metric spaces and propose estimators that can accommodate sparse and/or irregular designs. Consistency results are demonstrated for sample estimates of appropriate population targets along with the corresponding rates of convergence. The proposed models are illustrated with human mortality data and resting state functional Magnetic Resonance Imaging data (fMRI) as responses.

This article develops statistical methods for testing the equality of two distributions based on two independent samples generated in some separable metric space. Such methods are broadly applicable in identifying similarity or distinction of two complicated data sets (e.g., high-dimensional data or functional data) collected in a wide range of research or industry areas, including biology, bioinformatics, medicine, material science, among others. Recently a so-called maximum mean discrepancy (MMD) based approach for the above two-sample problem has been proposed, resulting in several interesting tests. However, the main theoretical and numerical results of these MMD based tests depend on the very restricted assumption that the two samples have equal sample sizes. In addition, these tests are generally implemented via permutation when the equal sample size assumption is violated. In real data analysis, this equal sample size assumption is hardly satisfied, and dropping away some of the observations often means the loss of priceless information. It is also of interest to know if an MMD-based test can be conducted generally without using permutation. In this paper, we further study this MMD based approach with the equal sample size assumption removed. We establish the asymptotic null and alternative distributions of the MMD test statistic and its root-n consistency. We propose methods for approximating the null distribution, resulting in easy and quick implementation. Numerical experiments based on artificial data and two real data sets from two different areas of applications demonstrate that in terms of control of the type I error level and power, the resulting tests perform better or no worse than several existing competitors.

The Receiver Operating Characteristic (ROC) curve is a useful tool to measure the classification capability of a continuous variable to assess the accuracy of a medical test that distinguishes between two conditions. Sometimes, covariates related to the diagnostic variable may increase the discriminating power of the ROC curve. Due to the lack of stability of classical ROC curves estimators to outliers, we introduce a procedure to obtain robust estimators in presence of covariates. The considered proposal focusses on a semiparametric approach which robustly fits a location-scale regression model to the diagnostic variable and considers robust adaptive empirical estimators of the regression residuals. The uniform consistency of the proposal is derived under mild assumptions. A Monte Carlo study is carried out to compare the performance of the robust proposed estimators with the classical ones both, in clean and contaminated samples. A real data set is also analysed.

In this paper, we consider an estimation problem about the tensor coefficient in a tensor regression model with multiple and unknown change-points. We generalize some recent findings in five ways. First, the problem studied is more general than the one in context of a matrix parameter with multiple change-points. Second, we develop asymptotic results of the tensor estimators in the context of a tensor regression with unknown change-points. Third, we construct a class of shrinkage tensor estimators that encompasses the unrestricted estimator (UE) and the restricted estimator (RE). Fourth, we generalize some identities which are crucial in deriving the asymptotic distributional risk (ADR) of the tensor estimators. Fifth, we show that the proposed shrinkage estimators perform better than the UE. The additional novelty of the established results consists in the fact that the dependence structure of the errors is as weak as that of an ${\mathcal{L}}^2$-mixingale. Finally, the theoretical results are corroborated by the simulation findings and our methods are applied to analyse MRI and fMRI datasets.

Covariance functions over generalized networks have been explored to a very limited extent. We consider nested spatial or space-time covariance models, where space is a generalized network, and where time can be linear (the real line) or circular. We show sufficient conditions allowing preservation of positive semidefiniteness when at least one of the weights involved in the linear combination is negative. Several examples illustrate our findings. In particular, we show nested constructions for Euclidean trees with a finite number of leaves involving basic covariance functions with different scale parameters or different compact supports. We also provide criteria that allow one to build space-time models through half spectral modeling on graphs cross linear or circular time.

In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with $\sqrt{n}$-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodness-of-fit and shape constraints.

We consider a class of multidimensional inhomogeneous diffusions whose drift coefficient depends on a multidimensional unknown parameter. Under some appropriate assumptions, we prove the local asymptotic mixed normality property for the drift parameter from high-frequency observations when the length of the observation window tends to infinity. To obtain the result, we use the Malliavin calculus techniques and the change of measures. Our approach is applicable for both ergodic and non-ergodic diffusions.

We study nonparametric estimation for the partially conditional average treatment effect, defined as the treatment effect function over an interested subset of confounders. We propose a double kernel weighting estimator where the weights aim to control the balancing error of any function of the confounders from a reproducing kernel Hilbert space after kernel smoothing over the interested subset of variables. In addition, we present an augmented version of our estimator which can incorporate the estimation of outcome mean functions. Based on the representer theorem, gradient-based algorithms can be applied for solving the corresponding infinite-dimensional optimization problem. Asymptotic properties are studied without any smoothness assumptions for the propensity score function or the need for data splitting, relaxing certain existing stringent assumptions. The numerical performance of the proposed estimator is demonstrated by a simulation study and an application to the effect of a mother’s smoking on a baby’s birth weight conditioned on the mother’s age.

This paper presents a theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov chain is constrained over an underlying graph so that states are viewed as vertices, and the transition between two states can have positive probability only in the presence of an edge connecting them. The analysis focuses on the relevant case of support of the target distribution not connected in the graph. Some general arguments on the speed of convergence are also carried out.

Most time series observed in practice exhibit time-varying trend (first-order) and autocovariance (second-order) behaviour. Differencing is a commonly-used technique to remove the trend in such series, in order to estimate the time-varying second-order structure (of the differenced series). However, often we require inference on the second-order behaviour of the original series, for example, when performing trend estimation. In this article, we propose a method, using differencing, to jointly estimate the time-varying trend and second-order structure of a nonstationary time series, within the locally stationary wavelet modelling framework. We develop a wavelet-based estimator of the second-order structure of the original time series based on the differenced estimate, and show how this can be incorporated into the estimation of the trend of the time series. We perform a simulation study to investigate the performance of the methodology, and demonstrate the utility of the method by analysing data examples from environmental and biomedical science.

We consider a high-dimensional monotone single index model (hdSIM), which is a semiparametric extension of a high-dimensional generalize linear model (hdGLM), where the link function is unknown, but constrained with monotone non-decreasing shape. We develop a scalable projection-based iterative approach, the “Sparse Orthogonal Descent Single-Index Model” (SOD-SIM), which alternates between sparse-thresholded orthogonalized “gradient-like” steps and isotonic regression steps to recover the coefficient vector. Our main contribution is that we provide finite sample estimation bounds for both the coefficient vector and the link function in high-dimensional settings under very mild assumptions on the design matrix X, the error term ϵ, and their dependence. The convergence rate for the link function matches the low-dimensional isotonic regression minimax rate up to some poly-log terms ($n^{-1∕3}$). The convergence rate for the coefficients is also $n^{-1∕3}$ up to some poly-log terms. This method can be applied to many real data problems, including GLMs with mis-specified link, classification with mislabeled data, and classification with positive-unlabeled (PU) data. We study the performance of this method via both numerical studies and also an application on a PU data example.

A common approach to detect multiple changepoints is to minimise a measure of data fit plus a penalty that is linear in the number of changepoints. This paper shows that the general finite sample behaviour of such a method can be related to its behaviour when analysing data with either none or one changepoint. This property results in simpler conditions for verifying whether the method will consistently estimate the number and locations of the changepoints. We apply and demonstrate the usefulness of these simple conditions for a range of changepoint problems. Our new results include a weaker requirement on the choice of penalty to have consistency in a change-in-slope model; and the first results for the accuracy of recently-proposed methods for detecting spikes.

One classical canon of statistics is that large models are prone to overfitting, and model selection procedures are necessary for high dimensional data. However, many overparameterized models, such as neural networks, perform very well in practice, although they are often trained with simple online methods and regularization. The empirical success of overparameterized models, which is often known as benign overfitting, motivates us to have a new look at the statistical generalization theory for online optimization. In particular, we present a general theory on the excess risk of stochastic gradient descent (SGD) solutions for both convex and locally non-convex loss functions. We further discuss data and model conditions that lead to a “low effective dimension”. Under these conditions, we show that the excess risk either does not depend on the ambient dimension p or depends on p via a poly-logarithmic factor. We also demonstrate that in several widely used statistical models, the “low effective dimension” arises naturally in overparameterized settings. The studied statistical applications include both convex models such as linear regression and logistic regression and non-convex models such as M-estimator and two-layer neural networks.

The cost of both generalized least squares (GLS) and Gibbs sampling in a crossed random effects model can easily grow faster than $N^{3∕2}$ for N observations. Ghosh et al. (2022) develop a backfitting algorithm that reduces the cost to $O(N)$. Here we extend that method to a generalized linear mixed model for logistic regression. We use backfitting within an iteratively reweighted penalized least squares algorithm. The specific approach is a version of penalized quasi-likelihood due to Schall (1991). A straightforward version of Schall’s algorithm would also cost more than $N^{3∕2}$ because it requires the trace of the inverse of a large matrix. We approximate that quantity at cost $O(N)$ and prove that this substitution makes an asymptotically negligible difference. Our backfitting algorithm also collapses the fixed effect with one random effect at a time in a way that is analogous to the collapsed Gibbs sampler of Papaspiliopoulos et al. (2020). We use a symmetric operator that facilitates efficient covariance computation. We illustrate our method on a real dataset from Stitch Fix. By properly accounting for crossed random effects we show that a naive logistic regression could underestimate sampling variances by several hundred fold.

We consider an analysis of variance type problem, where the sample observations are random elements in an infinite dimensional space. This scenario covers the case, where the observations are random functions. For such a problem, we propose a test based on spatial signs. We develop an asymptotic implementation as well as a bootstrap implementation and a permutation implementation of this test and investigate their size and power properties. We compare the performance of our test with that of several mean based tests of analysis of variance for functional data studied in the literature. Interestingly, our test not only outperforms the mean based tests in several non-Gaussian models with heavy tails or skewed distributions, but in some Gaussian models also. Further, we also compare the performance of our test with the mean based tests in several models involving contaminated probability distributions. Finally, we demonstrate the performance of these tests in three real datasets: a Canadian weather dataset, a spectrometric dataset on chemical analysis of meat samples and a dataset on orthotic measurements on volunteers.

A nonparametric regression setting is considered with a real-valued covariate and responses from a metric space. One may approach this setting via Fréchet regression, where the value of the regression function at each point is estimated via a Fréchet mean calculated from an estimated objective function. A second approach is geodesic regression, which builds upon fitting geodesics to observations by a least squares method. These approaches are applied to transform two of the most important nonparametric regression estimators in statistics to the metric setting – the local linear regression estimator and the orthogonal series projection estimator. The resulting procedures consist of known estimators as well as new methods. We investigate their rates of convergence in a general setting and compare their performance in a simulation study on the sphere.

Mixture of experts (MoE) are a popular class of statistical and machine learning models that have gained attention over the years due to their flexibility and efficiency. In this work, we consider Gaussian-gated localized MoE (GLoME) and block-diagonal covariance localized MoE (BLoME) regression models to present nonlinear relationships in heterogeneous data with potential hidden graph-structured interactions between high-dimensional predictors. These models pose difficult statistical estimation and model selection questions, both from a computational and theoretical perspective. This paper is devoted to the study of the problem of model selection among a collection of GLoME or BLoME models characterized by the number of mixture components, the complexity of Gaussian mean experts, and the hidden block-diagonal structures of the covariance matrices, in a penalized maximum likelihood estimation framework. In particular, we establish non-asymptotic risk bounds that take the form of weak oracle inequalities, provided that lower bounds for the penalties hold. The good empirical behavior of our models is then demonstrated on synthetic and real datasets.

A major task in Functional Time Series Analysis is measuring the dependence within and between processes for which lagged covariance and cross-covariance operators have proven to be a practical tool in well-established spaces. This article focuses on estimating these operators of processes in Cartesian products of abstract Hilbert spaces. We derive precise asymptotic results for the estimation errors for fixed and increasing lag and Cartesian powers under very mild conditions, presumably even under the mildest that can be assumed, establish estimators for the principal components, and conduct a simulation study.

We consider the Graph Ornstein-Uhlenbeck (GrOU) process observed on a non-uniform discrete timegrid and introduce discretised maximum likelihood estimators with parameters specific to the whole graph or specific to each component of the graph. Under a high-frequency sampling scheme, we study the asymptotic behaviour of those estimators as the mesh size of the observation grid goes to zero. We prove two stable central limit theorems to the same distribution as in the continuously-observed case under both finite and infinite jump activity for the Lévy driving noise. In addition to providing the consistency of the estimators, the stable convergence allows us to consider probabilistic sparse inference procedures on the edges themselves when a graph structure is not explicitly available. It also preserves its asymptotic properties. In particular, we also show the asymptotic normality and consistency of an Adaptive Lasso scheme. We apply the new estimators to wind capacity factor measurements, i.e. the ratio between the wind power produced locally compared to its rated peak power, across fifty locations in Northern Spain and Portugal. We compare those estimators to the standard least squares estimator through a simulation study extending known univariate results across graph configurations, noise types and amplitudes.

An important limitation of standard multiple testing procedures is that the null distribution should be known. Here, we consider a null distribution-free approach for multiple testing in the following semi-supervised setting: the user does not know the null distribution, but has at hand a sample drawn from this null distribution. In practical situations, this null training sample (NTS) can come from previous experiments, from a part of the data under test, from specific simulations, or from a sampling process. In this work, we present theoretical results that handle such a framework, with a focus on the false discovery rate (FDR) control and the Benjamini-Hochberg (BH) procedure. First, we provide upper and lower bounds for the FDR of the BH procedure based on empirical p-values, called here the semi-supervised BH procedure. These bounds match when $\mathit{\alpha }(n+1)∕m$ is an integer, where n is the NTS sample size and m is the number of tests. Second, we give a power analysis for that procedure suggesting that it mimics an oracle power when n is sufficiently large in front of m; namely $n\gtrsim m∕(max(1,k))$, where k denotes the number of “detectable” alternatives. Third, to complete the picture, we also present a negative result that evidences an intrinsic transition phase to the general semi-supervised multiple testing problem and shows that the semi-supervised BH method is optimal in the sense that its performance boundary follows this transition phase. Our theoretical properties are supported by numerical experiments, which also show that the delineated boundary is of correct order without further tuning any constant. Finally, we demonstrate that our work provides a theoretical ground for standard practice in astronomical data analysis, and in particular for the procedure proposed in Mary et al. (2020) for galaxy detection.

We consider the problem of detecting a general sparse mixture and obtain an explicit characterization of the phase transition under some conditions, generalizing the univariate results of Cai and Wu. Additionally, we provide a sufficient condition for the adaptive optimality of a Higher Criticism type testing statistic formulated by Gao and Ma. In the course of establishing these results, we offer a unified perspective through the large deviations theory. The phase transition and adaptive optimality we establish are direct consequences of the large deviation principle of the normalized log-likelihood ratios between the null and the signal distributions.

The ideal probabilistic forecast for a random variable Y based on an information set F is the conditional distribution of Y given F. In the context of point forecasts aiming to specify a functional T such as the mean, a quantile or a risk measure, the ideal point forecast is the respective functional applied to the conditional distribution. This paper provides a theoretical justification why this ideal forecast is actually a forecast, that is, an F-measurable random variable. To that end, the appropriate notion of measurability of T is clarified and this measurability is established for a large class of practically relevant functionals, including elicitable ones. More generally, the measurability of T implies the measurability of any point forecast which arises by applying T to a probabilistic forecast. Similar measurability results are established for proper scoring rules, the main tool to evaluate the predictive accuracy of probabilistic forecasts.

We consider a data matrix $\mathbf{X}:={\mathbf{C}}_N^{1∕2}\mathbf{Z}{\mathbf{R}}_M^{1∕2}$ from a multivariate stationary process with a separable covariance function, where ${\mathbf{C}}_N$ is a $N\times N$ positive semi-definite matrix, Z a $N\times M$ random matrix of uncorrelated standardized white noise, and ${\mathbf{R}}_M$ a $M\times M$ Toeplitz matrix. Under the assumption of long range dependence (LRD), we re-examine the consistency of two toeplitzifized estimators ${\hat{\mathbf{R}}}_M$ (unbiased) and ${\hat{\mathbf{R}}}_M^b$ (biased) for ${\mathbf{R}}_M$, which are known to be norm consistent with ${\mathbf{R}}_M$ when the process is short range dependent (SRD). However in the LRD case, some simulations suggest that the norm consistency does not hold in general for both estimators. Instead, a weaker ratio consistency is established for the unbiased estimator ${\hat{\mathbf{R}}}_M$, and a further weaker ratio LSD consistency is established for the biased estimator ${\hat{\mathbf{R}}}_M^b$. The main result leads to a consistent whitening procedure on the original data matrix X, which is further applied to two real world questions, one is a signal detection problem, and the other is PCA on the space covariance ${\mathbf{C}}_N$ to achieve a noise reduction and data compression.

We propose nonparametric estimators for the second-order central moments of possibly anisotropic spherical random fields, within a functional data analysis context. We consider a measurement framework where each random field among an identically distributed collection of spherical random fields is sampled at a few random directions, possibly subject to measurement error. The collection of random fields could be i.i.d. or serially dependent. Though similar setups have already been explored for random functions defined on the unit interval, the nonparametric estimators proposed in the literature often rely on local polynomials, which do not readily extend to the (product) spherical setting. We therefore formulate our estimation procedure as a variational problem involving a generalized Tikhonov regularization term. The latter favours smooth covariance/autocovariance functions, where the smoothness is specified by means of suitable Sobolev-like pseudo-differential operators. Using the machinery of reproducing kernel Hilbert spaces, we establish representer theorems that fully characterize the form of our estimators. We determine their uniform rates of convergence as the number of random fields diverges, both for the dense (increasing number of spatial samples) and sparse (bounded number of spatial samples) regimes. We moreover demonstrate the computational feasibility and practical merits of our estimation procedure in a simulation setting, assuming a fixed number of samples per random field. Our numerical estimation procedure leverages the sparsity and second-order Kronecker structure of our setup to reduce the computational and memory requirements by approximately three orders of magnitude compared to a naive implementation would require.

Statistical inference for normal mixture models with unknown number of components has long been challenging due to the issues of nonidentifiability, degenerated Fisher matrix, and boundary parameters. In this paper, a penalized likelihood estimation procedure is proposed for mixtures of normals with unknown number of components to achieve both the order selection consistency and the root-n convergence rate for the component parameters estimators. We show that the proposed new estimator could avoid being trapped in certain degenerated regions of the nonidentifiable subset of the parameter space for over-fitted normal mixture models so that a regular asymptotic quadratic Taylor expansion of the mixture log-likelihood could be derived. With a suitable penalty function on mixing proportions, the new estimator is proved to be consistent on the order selection, and have an asymptotic normal distribution. Our derived sparsity conditions also reveal some surprising but interesting differences among some commonly used penalty functions and explain why the performance of some popularly used penalty functions, such as Lasso and SCAD, provide unsatisfactory results in the order selection. Extensive simulations and a real data analysis are conducted to demonstrate the effectiveness of the newly proposed estimator.

Linear mixed models are a versatile statistical tool to study data by accounting for fixed effects and random effects from multiple sources of variability. In many situations, a large number of candidate fixed effects is available and it is of interest to select a parsimonious subset of those being effectively relevant for predicting the response variable. Variational approximations facilitate fast approximate Bayesian inference for the parameters of a variety of statistical models, including linear mixed models. However, for models having a high number of fixed or random effects, simple application of standard variational inference principles does not lead to fast approximate inference algorithms, due to the size of model design matrices and inefficient treatment of sparse matrix problems arising from the required approximating density parameters updates.

We illustrate how recently developed streamlined variational inference procedures can be generalized to make fast and accurate inference for the parameters of linear mixed models with nested random effects and global-local priors for Bayesian fixed effects selection. Our variational inference algorithms achieve convergence to the same optima of their standard implementations, although with significantly lower computational effort, memory usage and time, especially for large numbers of random effects. Using simulated and real data examples, we assess the quality of automated procedures for fixed effects selection that are free from hyperparameters tuning and only rely upon variational posterior approximations. Moreover, we show high accuracy of variational approximations against model fitting via Markov Chain Monte Carlo sampling.

Testing the simplifying assumption in high-dimensional vine copulas is a difficult task. Tests must be based on estimated observations and check constraints on high-dimensional distributions. So far, corresponding tests have been limited to single conditional copulas with a low-dimensional set of conditioning variables. We propose a novel testing procedure that is computationally feasible for high-dimensional data sets and that exhibits a power that decreases only slightly with the dimension. By discretizing the support of the conditioning variables and incorporating a penalty in the test statistic, we mitigate the curse of dimensionality by looking for the possibly strongest deviation from the simplifying assumption. The use of a decision tree renders the test computationally feasible for large dimensions. We derive the asymptotic distribution of the test and analyze its finite sample performance in an extensive simulation study. An application of the test to four real data sets is provided.

A framework for hypothesis testing of functional restrictions against general alternatives is proposed. The parameter space is a subset of a reproducing kernel Hilbert space (RKHS). The null hypothesis does not necessarily define a parametric model. The test allows us to deal with possibly infinite dimensional nuisance parameters. The methodology is based on a moment equation similar in spirit to the construction of the efficient score in semiparametric statistics. The feasible version of such moment equation requires to consistently estimate projections in the space of RKHS. A tractable asymptotic theory is established for this problem. Simulation results show that the finite sample performance of the test is consistent with the asymptotic results and that ignoring the effect of nuisance parameters highly distorts the size of the tests.

The density ratio model (DRM) provides a flexible and useful platform for combining information from multiple sources. In this paper, we consider statistical inference under two-sample DRMs with additional parameters defined through and/or additional auxiliary information expressed as estimating equations. We examine the asymptotic properties of the maximum empirical likelihood estimators (MELEs) of the unknown parameters in the DRMs and/or defined through estimating equations, and establish the chi-square limiting distributions for the empirical likelihood ratio (ELR) statistics. We show that the asymptotic variance of the MELEs of the unknown parameters does not decrease if one estimating equation is dropped. Similar properties are obtained for inferences on the cumulative distribution function and quantiles of each of the populations involved. We also propose an ELR test for the validity and usefulness of the auxiliary information. Simulation studies show that correctly specified estimating equations for the auxiliary information result in more efficient estimators and shorter confidence intervals. Two real examples are used for illustrations.

Linear discriminant analysis (LDA) is an important classification tool in statistics and machine learning. This paper investigates the varying coefficient LDA model for dynamic data, with Bayes’ discriminant direction being a function of some exposure variable to address the heterogeneity. We propose a new least-square estimation method based on the B-spline approximation. The data-driven discriminant procedure is more computationally efficient than the dynamic linear programming rule [21]. We also establish the convergence rates for the corresponding estimation error bound and the excess misclassification risk. The estimation error in ${\mathrm{L}}_2$ distance is optimal for the low-dimensional regime and is near optimal for the high-dimensional regime. Numerical experiments on synthetic data and real data both corroborate the superiority of our proposed classification method.

We study a high-dimensional linear regression model in a semi-supervised setting, where for many observations only the vector of covariates X is given with no responses Y. We do not make any sparsity assumptions on the vector of coefficients, nor do we assume normality of the covariates. We aim at estimating the signal level, i.e., the amount of variation in the response that can be explained by the set of covariates. We propose an estimator, which is unbiased, consistent, and asymptotically normal. This estimator can be improved by adding zero-estimators arising from the unlabeled data. Adding zero-estimators does not affect the bias and potentially can reduce the variance. We further present an algorithm based on our approach that improves any given signal level estimator. Our theoretical results are demonstrated in a simulation study.

We build a general framework which establishes a one-to-one correspondence between species abundance distribution (SAD) and species accumulation curve (SAC). The appearance rates of the species and the appearance times of individuals of each species are modeled as Poisson processes. The number of species can be finite or infinite. Hill numbers are extended to the framework. We introduce a linear derivative ratio family of models, ${\mathrm{LDR}}_1$, of which the ratio of the first and the second derivatives of the expected SAC is a linear function. A D1/D2 plot is proposed to detect this linear pattern in the data. By extrapolation of the curve in the D1/D2 plot, a species richness estimator that extends Chao1 estimator is introduced. The SAD of ${\mathrm{LDR}}_1$ is the Engen’s extended negative binomial distribution, and the SAC encompasses several popular parametric forms including the power law. Family ${\mathrm{LDR}}_1$ is extended in two ways: ${\mathrm{LDR}}_2$ which allows species with zero detection probability, and ${\mathrm{RDR}}_1$ where the derivative ratio is a rational function. Real data are analyzed to demonstrate the proposed methods. We also consider the scenario where we record only a few leading appearance times of each species. We show how maximum likelihood inference can be performed when only the empirical SAC is observed, and elucidate its advantages over the traditional curve-fitting method.

Recursive algorithms, based upon the nested structure of Toeplitz covariance matrices arising from stationary processes, are presented for the efficient computation of multi-step ahead forecast error covariances for nonstationary vector time series. Further, we discuss time reversal to forecast the past, and a filtering algorithm for imputation of missing values. These quantities are required to quantify multi-step ahead forecast error and signal extraction error. An information filter is presented, which provides imputations for arbitrary patterns of missing values (such as ragged edge patterns occurring in mixed frequency data). The methods are applied to multivariate retail data exhibiting trend dynamics and seasonality.

Importance sampling (IS) is a Monte Carlo technique that relies on weighted samples, simulated from a proposal distribution, to estimate intractable integrals. The quality of the estimators improves with the number of samples. However, for achieving a desired quality of estimation, the required number of samples is unknown and depends on the quantity of interest, the estimator, and the chosen proposal. We present a sequential stopping rule that terminates simulation when the overall variability in estimation is relatively small. The proposed methodology closely connects to the idea of an effective sample size in IS and overcomes crucial shortcomings of existing metrics, e.g., it acknowledges multivariate estimation problems. Our stopping rule retains asymptotic guarantees and provides users a clear guideline on when to stop the simulation in IS.

This paper develops asymptotic normality results for individual coordinates of robust M-estimators with convex penalty in high-dimensions, where the dimension p is at most of the same order as the sample size n, i.e, $p∕n\le \mathit{\gamma }$ for some fixed constant $\mathit{\gamma }>0$. The asymptotic normality requires a bias correction and holds for most coordinates of the M-estimator for a large class of loss functions including the Huber loss and its smoothed versions regularized with a strongly convex penalty.

The asymptotic variance that characterizes the width of the resulting confidence intervals is estimated with data-driven quantities. This estimate of the variance adapts automatically to low ($p∕n\to 0)$ or high ($p∕n\le \mathit{\gamma }$) dimensions and does not involve the proximal operators seen in previous works on asymptotic normality of M-estimators. For the Huber loss, the estimated variance has a simple expression involving an effective degrees-of-freedom as well as an effective sample size. The case of the Huber loss with Elastic-Net penalty is studied in details and a simulation study confirms the theoretical findings. The asymptotic normality results follow from Stein formulae for high-dimensional random vectors on the sphere developed in the paper which are of independent interest.

The mean shift (MS) algorithm is a nonparametric method used to cluster sample points and find the local modes of kernel density estimates, using an idea based on iterative gradient ascent. In this paper we develop a mean-shift-inspired algorithm to estimate the maxima of regression functions and partition the sample points in the input space. We prove convergence of the sequences generated by the algorithm and derive the rates of convergence of the estimated local maxima for the underlying regression model. We also demonstrate the utility of the algorithm for data-enabled discovery through an application on biomolecular structure data.

Optimal design of experiments for correlated processes is an increasingly relevant and active research topic. Present methods have restricted possibilities to judge their quality. To fill this gap, we complement the virtual noise approach by a convex formulation leading to an equivalence theorem comparable to the uncorrelated case and to an algorithm giving an upper performance bound against which alternative design methods can be judged. Moreover, a method for generating exact designs follows naturally. We exclusively consider estimation problems on a finite design space with a fixed number of elements. A comparison on some classical examples from the literature as well as a real application is provided.

We study the weak convergence of conditional empirical copula processes indexed by general families of conditioning events that have non zero probabilities. Moreover, we also study the case where the conditioning events are chosen in a data-driven way. The validity of several bootstrap schemes is stated, including the exchangeable bootstrap. We define general multivariate measures of association, possibly given some fixed or random conditioning events. By applying our theoretical results, we prove the asymptotic normality of the estimators of such measures. We illustrate our results with financial data.

We consider the problem of finding the matching map between two sets of d-dimensional vectors from noisy observations, where the second set contains outliers.The matching map is then an injection, which can be consistently detected only if the vectors of the second set are well separated. The main result shows that, in the high-dimensional setting, a detection region of unknown injection may be characterized by the sets of vectors for which the inlier-inlier distance is of order at least $d^{1∕4}$ and the inlier-outlier distance is of order at least $d^{1∕2}$. These rates are achieved using the matching minimizing the sum of logarithms of distances between matched pairs of points. We also prove lower bounds establishing optimality of these rates. Finally, we report the results of numerical experiments on both synthetic and real world data that illustrate our theoretical results and provide further insight into the properties of the algorithms studied in this work.

We consider the problem of estimating the conditional distribution $P(Y\in A|X)$ of a functional data object $Y=(Y(t):t\in [0,1])$ in the space of continuous functions, given covariates X in a general space and assuming that Y and X are related by a functional linear regression model. Two estimation methods are proposed, based on either the empirical distribution of the estimated model residuals, or fitting functional parametric models to the model residuals. We show that consistent estimation can be achieved under relatively mild assumptions. We exemplify a general class of sets A specifying path properties of Y that are of interest in applications. The proposed methods are studied in several simulation experiments, and data analyses of electricity price and pollution curves.

The Pitman-Yor process is a random probability distribution, that can be used as a prior distribution in a nonparametric Bayesian analysis. The process is of species sampling type and generates discrete distributions, which yield of the order $n^{\mathit{\sigma }}$ different values (“species”) in a random sample of size n, if the type σ is positive. Thus this type parameter can be set to target true distributions of various levels of discreteness, making the Pitman-Yor process an interesting prior in this case. It was previously shown that the resulting posterior distribution is consistent if and only if the true distribution of the data is discrete. In this paper we derive the distributional limit of the posterior distribution, in the form of a (corrected) Bernstein-von Mises theorem, which previously was known only in the continuous, inconsistent case. It turns out that the Pitman-Yor posterior distribution has good behaviour if the true distribution of the data is discrete with atoms that decrease not too slowly. Credible sets derived from the posterior distribution provide valid frequentist confidence sets in this case. For a general discrete distribution, the posterior distribution, although consistent, may contain a bias which does not converge to zero at the $\sqrt{n}$ rate and invalidates posterior inference. We propose a bias correction that solves this problem. We also consider the effect of estimating the type parameter from the data, both by empirical Bayes and full Bayes methods. In a small simulation study we illustrate that without bias correction the coverage of credible sets can be arbitrarily low, also for some discrete distributions.

Training a classifier under non-convex constraints has gotten increasing attention in the machine learning community thanks to its wide range of applications such as algorithmic fairness and class-imbalanced classification. However, several recent works addressing non-convex constraints have only focused on simple models such as logistic regression or support vector machines. Neural networks, one of the most popular models for classification nowadays, are precluded and lack theoretical guarantees. In this work, we show that overparameterized neural networks could achieve a near-optimal and near-feasible solution of non-convex constrained optimization problems via the project stochastic gradient descent. Our key ingredient is the no-regret analysis of online learning for neural networks in the overparameterization regime, which may be of independent interest in online learning applications.

We study the performance of shape-constrained methods for evaluating immune response profiles from early-phase vaccine trials. The motivating problem for this work involves quantifying and comparing the IgG binding immune responses to the first and second variable loops (V1V2 region) arising in HVTN 097 and HVTN 100 HIV vaccine trials. We consider unimodal and log-concave shape-constrained methods to compare the immune profiles of the two vaccines, which is reasonable because the data support that the underlying densities of the immune responses could have these shapes. To this end, we develop novel shape-constrained tests of stochastic dominance and shape-constrained plug-in estimators of the squared Hellinger distance between two densities. Our techniques are either tuning parameter free, or rely on only one tuning parameter, but their performance is either better (the tests of stochastic dominance) or comparable with the nonparametric methods (the estimators of the squared Hellinger distance). The minimal dependence on tuning parameters is especially desirable in clinical contexts where analyses must be prespecified and reproducible. Our methods are supported by theoretical results and simulation studies.

Let $f(y\mid \mathit{\theta }),\mathit{\theta }\in \mathrm{\Omega }$ be a parametric family, $\mathrm{\eta }(\mathit{\theta })$ a given function, and G an unknown mixing distribution. It is desired to estimate $E_G(\mathrm{\eta }(\mathit{\theta }))\equiv {\mathrm{\eta }}_G$ based on independent observations $Y_1,\mathrm{...},Y_n$, where $Y_i\sim f(y\mid {\mathit{\theta }}_i)$, and ${\mathit{\theta }}_i\sim G$ are iid.

We explore the Generalized Maximum Likelihood Estimators (GMLE) for this problem. Some basic properties and representations of those estimators are shown. In particular we suggest a new perspective, of the weak convergence result by [14], with implications to a corresponding setup in which ${\mathit{\theta }}_1,\mathrm{...},{\mathit{\theta }}_n$ are fixed parameters. We also relate the above problem, of estimating ${\mathrm{\eta }}_G$, to nonparametric empirical Bayes estimation under a squared loss.

Applications of GMLE to sampling problems are presented. The performance of the GMLE is demonstrated both in simulations and through a real data example.

We study the problem of identifying the set of active variables, termed in the literature as variable selection or multiple hypothesis testing, depending on the pursued criteria. For a general robust setting of non-normal, possibly dependent observations and a generalized notion of active set, we propose a procedure that is used simultaneously for the both tasks, variable selection and multiple testing. The procedure is based on the risk hull minimization method, but can also be obtained as a result of an empirical Bayes approach or a penalization strategy. We address its quality via various criteria: the Hamming risk, FDR, FPR, FWER, NDR, FNR, and various multiple testing risks, e.g., MTR=FDR+NDR; and discuss a weak optimality of our results. Finally, we introduce and study, for the first time, the uncertainty quantification problem in the variable selection and multiple testing context in our robust setting.

In this paper, we consider the linear regression model $Y=SX+\mathit{\epsilon }$ with functional regressors and responses. We develop new inference tools to quantify deviations of the true slope S from a hypothesized operator $S_0$ with respect to the Hilbert–Schmidt norm $~S-S_0{~}^2$, as well as the prediction error $\mathbb{E}\Vert SX-S_{0}X{\Vert }^2$. Our analysis is applicable to functional time series and based on asymptotically pivotal statistics. This makes it particularly user-friendly, because it avoids the choice of tuning parameters inherent in long-run variance estimation or bootstrap of dependent data. We also discuss two sample problems as well as change point detection. Finite sample properties are investigated by means of a simulation study.

Mathematically, our approach is based on a sequential version of the popular spectral cut-off estimator ${\hat{S}}_N$ for S. We prove that (sequential) plug-in estimators of the deviation measures are $\sqrt{N}$-consistent and satisfy weak invariance principles. These results rest on the smoothing effect of $L^2$-norms, that we exploit by a new proof-technique, the smoothness shift, which has potential applications in other fields.

This paper continues the line of research initiated in Liu, Sarkar and Zhao (2016) on developing a novel framework for multiple testing of hypotheses grouped in a one-way classified form using hypothesis-specific local false discovery rates (Lfdr’s). It is built on an extension of the standard two-class mixture model from single to multiple groups, defining hypothesis-specific Lfdr as a function of the conditional Lfdr for the hypothesis given that it is within an important group and the Lfdr for the group itself and involving a new parameter that measures grouping effect. This definition captures the underlying group structure for the hypotheses belonging to a group more effectively than the standard two-class mixture model. Two new Lfdr based methods, possessing meaningful optimalities, are produced in their oracle forms. One, designed to control false discoveries across the entire collection of hypotheses, is proposed as a powerful alternative to simply pooling all the hypotheses into a single group and using commonly used Lfdr based method under the standard single-group two-class mixture model. The other is proposed as an Lfdr analog of the method of Benjamini and Bogomolov (2014) for selective inference. It controls Lfdr based measure of false discoveries associated with selecting groups concurrently with controlling the average of within-group false discovery proportions across the selected groups. Simulation studies and real-data application show that our proposed methods are often more powerful than their relevant competitors.

With the deluge of digitized information in the Big Data era, massive datasets are becoming increasingly available for learning predictive models. However, in many practical situations, the poor control of the data acquisition processes may naturally jeopardize the outputs of machine learning algorithms, and selection bias issues are now the subject of much attention in the literature. The present article investigates how to extend Empirical Risk Minimization, the principal paradigm in statistical learning, when training observations are generated from biased models, i.e., from distributions that are different from that in the test/prediction stage, and absolutely continuous with respect to the latter. Precisely, we show how to build a “nearly debiased” training statistical population from biased samples and the related biasing functions, following in the footsteps of the approach originally proposed in [46]. Furthermore, we study from a nonasymptotic perspective the performance of minimizers of an empirical version of the risk computed from the statistical population thus created. Remarkably, the learning rate achieved by this procedure is of the same order as that attained in absence of selection bias. Beyond the theoretical guarantees, we also present experimental results supporting the relevance of the algorithmic approach promoted in this paper.

We study the class of dependence models for spatial data obtained from Cauchy convolution processes based on different types of kernel functions. We show that the resulting spatial processes have appealing tail dependence properties, such as tail dependence at short distances and independence at long distances with suitable kernel functions. We derive the extreme-value limits of these processes, study their smoothness properties, and detail some interesting special cases. To get higher flexibility at sub-asymptotic levels and separately control the bulk and the tail dependence properties, we further propose spatial models constructed by mixing a Cauchy convolution process with a Gaussian process. We demonstrate that this framework indeed provides a rich class of models for the joint modeling of the bulk and the tail behaviors. Our proposed inference approach relies on matching model-based and empirical summary statistics, and an extensive simulation study shows that it yields accurate estimates. We demonstrate our new methodology by application to a temperature dataset measured at 97 monitoring stations in the state of Oklahoma, US. Our results indicate that our proposed model provides a very good fit to the data, and that it captures both the bulk and the tail dependence structures accurately.

We consider the problem of constructing nonparametric undirected graphical models for high-dimensional functional data. Most existing statistical methods in this context assume either a Gaussian distribution on the vertices or linear conditional means. In this article, we provide a more flexible model which relaxes the linearity assumption by replacing it by an arbitrary additive form. The use of functional principal components offers an estimation strategy that uses a group lasso penalty to estimate the relevant edges of the graph. We establish statistical guarantees for the resulting estimators, which can be used to prove consistency if the dimension and the number of functional principal components diverge to infinity with the sample size. We also investigate the empirical performance of our method through simulation studies and a real data application

We consider functional data which are measured on a discrete set of observation points. Often such data are measured with additional noise. We explore in this paper the factor structure underlying this type of data. We show that the latent signal can be attributed to the common components of a corresponding factor model and can be estimated accordingly, by borrowing methods from factor model literature. We also show that principal components, which play a key role in functional data analysis, can be accurately estimated by taking such a multivariate instead of a ‘functional’ perspective. In addition to the estimation problem, we also address testing of the null-hypothesis of iid noise. While this assumption is largely prevailing in the literature, we believe that it is often unrealistic and not supported by a residual analysis.

We consider statistical inference in the density estimation model using a tree–based Bayesian approach, with Optional Pólya trees as prior distribution. We derive near-optimal convergence rates for corresponding posterior distributions with respect to the supremum norm. For broad classes of Hölder–smooth densities, we show that the method automatically adapts to the unknown Hölder regularity parameter. We consider the question of uncertainty quantification by providing mathematical guarantees for credible sets from the obtained posterior distributions, leading to near–optimal uncertainty quantification for the density function, as well as related functionals such as the cumulative distribution function. The results are illustrated through a brief simulation study.

This paper analyzes various classes of processes associated with the tempered positive Linnik (TPL) distribution. We provide several subordinated representations of TPL Lévy processes and in particular establish a stochastic self-similarity property with respect to negative binomial subordination. In finite activity regimes we show that the explicit compound Poisson representations give raise to innovations following Mittag-Leffler type laws which are apparently new. We characterize two time-inhomogeneous TPL processes, namely the Ornstein-Uhlenbeck (OU) Lévy-driven processes with stationary distribution and the additive process determined by a TPL law. We finally illustrate how the properties studied come together in a multivariate TPL Lévy framework based on a novel negative binomial mixing methodology. Some potential applications are outlined in the contexts of statistical anti-fraud and financial modelling.

For testing conditional independence (CI) of a response Y and a predictor X given covariates Z, the model-X (MX) framework has been the subject of active methodological research, especially in the context of MX knockoffs and their application to genome-wide association studies. In this paper, we study the power of MX CI tests, yielding quantitative insights into the role of machine learning and providing evidence in favor of using likelihood-based statistics in practice. Focusing on the conditional randomization test (CRT), we find that its conditional mode of inference allows us to reformulate it as testing a point null hypothesis involving the conditional distribution of X. The Neyman-Pearson lemma implies that a likelihood-based statistic yields the most powerful CRT against a point alternative. We obtain a related optimality result for MX knockoffs. Switching to an asymptotic framework with arbitrarily growing covariate dimension, we derive an expression for the power of the CRT against local semiparametric alternatives in terms of the prediction error of the machine learning algorithm on which its test statistic is based. Finally, we exhibit a resampling-free test with uniform asymptotic Type-I error control under the assumption that only the first two moments of X given Z are known.

Over the last decades, various “non-linear” MCMC methods have arisen. While appealing for their convergence speed and efficiency, their practical implementation and theoretical study remain challenging. In this paper, we introduce a non-linear generalization of the Metropolis-Hastings algorithm to a proposal that depends not only on the current state, but also on its law. We propose to simulate this dynamics as the mean field limit of a system of interacting particles, that can in turn itself be understood as a generalisation of the Metropolis-Hastings algorithm to a population of particles. Under the double limit in number of iterations and number of particles we prove that this algorithm converges. Then, we propose an efficient GPU implementation and illustrate its performance on various examples. The method is particularly stable on multimodal examples and converges faster than the classical methods.

In this paper, we propose a robust estimator for the location function from multi-dimensional functional data. The proposed estimators are based on the deep neural networks with ReLU activation function. At the meanwhile, the estimators are less susceptible to outlying observations and model-misspecification. For any multi-dimensional functional data, we provide the uniform convergence rates for the proposed robust deep neural networks estimators. Simulation studies illustrate the competitive performance of the robust deep neural network estimators on regular data and their superior performance on data that contain anomalies. The proposed method is also applied to analyze 2D and 3D images of patients with Alzheimer’s disease obtained from the Alzheimer Disease Neuroimaging Initiative database.

Thanks to its favorable properties, the multivariate normal distribution is still largely employed for modeling phenomena in various scientific fields. However, when the number of components p is of the same asymptotic order as the sample size n, standard inferential techniques are generally inadequate to conduct hypothesis testing on the mean vector and/or the covariance matrix. Within several prominent frameworks, we propose then to draw reliable conclusions via a directional test. We show that under the null hypothesis the directional p-value is exactly uniformly distributed even when p is of the same order of n, provided that conditions for the existence of the maximum likelihood estimate for the normal model are satisfied. Extensive simulation results confirm the theoretical findings across different values of $p∕n$, and show that under the null hypothesis the directional test outperforms not only the usual first and higher-order finite-p solutions but also alternative methods tailored for high dimensional settings. Simulation results also indicate that the power performance of the different tests depends on the specific alternative hypothesis.

We consider the problem of recovering a k-sparse signal ${\mathit{\beta }}_0\in {\mathbb{R}}^p$ from noisy observations $\mathbf{y}=\mathbf{X}{\mathit{\beta }}_0+\mathbf{w}\in {\mathbb{R}}^n$. One of the most popular approaches is the $l_1$-regularized least squares, also known as LASSO. We analyze the mean square error of LASSO in the case of random designs in which each row of X is drawn from distribution $N(0,\mathrm{\Sigma })$ with general $\mathrm{\Sigma }$. We first derive the asymptotic risk of LASSO for $\mathbf{w}\ne 0$ in the limit of $n,p\to \infty$ with $n∕p\to \mathit{\delta }\in [0,\infty )$. We then examine conditions on n, p, and k for LASSO to exactly reconstruct ${\mathit{\beta }}_0$ in the noiseless case $\mathbf{w}=0$. A phase boundary ${\mathit{\delta }}_c=\mathit{\delta }(\mathit{ϵ})$ is precisely established in the phase space defined by $0\le \mathit{\delta },\mathit{ϵ}\le 1$, where $\mathit{ϵ}=k∕p$. Above this boundary, LASSO perfectly recovers ${\mathit{\beta }}_0$ with high probability. Below this boundary, LASSO fails to recover ${\mathit{\beta }}_0$ with high probability. While the values of the non-zero elements of ${\mathit{\beta }}_0$ do not have any effect on the phase transition curve, our analysis shows that ${\mathit{\delta }}_c$ does depend on the signed pattern of the nonzero values of ${\mathit{\beta }}_0$ for general $\mathrm{\Sigma }\ne {\mathbf{I}}_{p\times p}$. This is in sharp contrast to the previous phase transition results derived in i.i.d. case with $\mathrm{\Sigma }={\mathbf{I}}_{p\times p}$ where ${\mathit{\delta }}_c$ is completely determined by ϵ regardless of the distribution of ${\mathit{\beta }}_0$. Underlying our formalism is a recently developed efficient algorithm called approximate message passing (AMP) algorithm. We generalize the state evolution of AMP from i.i.d. case to general case with $\mathrm{\Sigma }\ne {\mathbf{I}}_{p\times p}$. Extensive computational experiments confirm that our theoretical predictions are consistent with simulation results on moderate size system.

We propose a theoretical study of two realistic estimators of conditional distribution functions using random forests. The estimation process uses the bootstrap samples generated from the original dataset when constructing the forest. Bootstrap samples are reused to define the first estimator, while the second uses the original sample, once the forest has been built. We prove that both proposed estimators of the conditional distribution functions are consistent uniformly a.s. To the best of our knowledge, it is the first proof of a.s. consistency (previous consistency results are in $L^2$ norm or in probability) and including the bootstrap part. The consistency result holds for a large class of functions, including additive models and products. The consistency of conditional quantiles estimators follows that of distribution functions estimators using standard arguments.

Depth functions have become increasingly powerful tools in non-parametric inference for multivariate data as they measure a degree of centrality of a point with respect to a distribution. A multivariate risk scenario is then represented by a depth-based lower level set of the risk factors, meaning that we consider a non-compact setting. The aim of this paper is to study the asymptotic behavior of level sets of a general multivariate depth function and a particular multivariate risk measure, the Covariate-Conditional-Tail-Expectation (CCTE) based on a depth function. More precisely, given a probability measure P on ${\mathbb{R}}^d$ and a depth function $D(\cdot ,P)$, we are interested in the α-lower level set $L_D(\mathit{\alpha }):=\left\{z\in {\mathbb{R}}^d:D(z,P)\le \mathit{\alpha }\right\}$. First, we present a plug-in approach in order to estimate $L_D(\mathit{\alpha })$, then we derive consistency of its estimator under some regularity conditions. In a second part, we provide a consistent estimator of the CCTE for a general depth function with a rate of convergence and we consider the particular case of Mahalanobis depth. Finally, a simulation study complements the performances of our estimator and an application on real data is presented.

The recently introduced framework of universal inference provides a new approach to constructing hypothesis tests and confidence regions that are valid in finite samples and do not rely on any specific regularity assumptions on the underlying statistical model. At the core of the methodology is a split likelihood ratio statistic, which is formed under data splitting and compared to a cleverly selected universal critical value. As this critical value can be very conservative, it is interesting to mitigate the potential loss of power by careful choice of the ratio according to which data are split. Motivated by this problem, we study the split likelihood ratio test under local alternatives and introduce the resulting class of noncentral split chi-square distributions. We investigate the properties of this new class of distributions and use it to numerically examine and propose an optimal choice of the data splitting ratio for tests of composite hypotheses of different dimensions.