In this paper, we use the class of Wasserstein metrics to study asymptotic properties of posterior distributions. Our first goal is to provide sufficient conditions for posterior consistency. In addition to the well-known Schwartz’s Kullback–Leibler condition on the prior, the true distribution and most probability measures in the support of the prior are required to possess moments up to an order which is determined by the order of the Wasserstein metric. We further investigate convergence rates of the posterior distributions for which we need stronger moment conditions. The required tail conditions are sharp in the sense that the posterior distribution may be inconsistent or contract slowly to the true distribution without these conditions. Our study involves techniques that build on recent advances on Wasserstein convergence of empirical measures. We apply the results to some examples including a Dirichlet process mixture prior and conduct a simulation study for further illustration.

In network data analysis, multivariate spatial autoregression (MSAR) models may be used to analyze the autocorrelation among multiple responses. With large-scale networks, the estimation for MSAR on the entire network is computationally expensive. In this case, the subsampling method could be adopted. This approach selects a sample of nodes and then uses the estimate based on the sample to approximate the estimate on the full data. However, traditional sampling methods cannot obtain unbiased parameter estimates. Considering the second-order friend information of sampled nodes, we propose the crawling subsampling (CS) method for a general framework. Thereafter, based on the sampled data only, we construct the least-squares objective function. Under certain conditions, the computational complexity of optimizing the objective function is linear with the sample size $n_s$. The identification condition for the parameters on the sampled network is theoretically provided. The sample size order requirement is provided, and the asymptotic properties of the least-squares estimators are investigated. The numerical results for the simulated and real data are presented to demonstrate the performance of the proposed CS method and least-squares estimator for the MSAR model.

We consider survival data from a population with cured subjects in the presence of mismeasured covariates. We use the mixture cure model to account for the individuals that will never experience the event and at the same time distinguish between the effect of the covariates on the cure probabilities and on survival times. In particular, for practical applications, it seems of interest to assume a logistic form of the incidence and a Cox proportional hazards model for the latency. To correct the estimators for the bias introduced by the measurement error, we use the simex algorithm, which is a very general simulation based method. It essentially estimates this bias by introducing additional error to the data and then recovers bias corrected estimators through an extrapolation approach. The estimators are shown to be consistent and asymptotically normally distributed when the true extrapolation function is known. We investigate their finite sample performance through a simulation study and apply the proposed method to analyse the effect of the prostate specific antigen (PSA) on patients with prostate cancer.

Kernel ridge regression (KRR) is a well-known and popular nonparametric regression approach with many desirable properties, including minimax rate-optimality in estimating functions that belong to common reproducing kernel Hilbert spaces (RKHS). The approach, however, is computationally intensive for large data sets, due to the need to operate on a dense $n\times n$ kernel matrix, where n is the sample size. Recently, various approximation schemes for solving KRR have been considered, and some analyzed. Some approaches such as Nyström approximation and sketching have been shown to preserve the rate optimality of KRR. In this paper, we consider the simplest approximation, namely, spectrally truncating the kernel matrix to its largest $r<n$ eigenvalues. We derive an exact expression for the maximum risk of this truncated KRR, over the unit ball of the RKHS. This result can be used to study the exact trade-off between the level of spectral truncation and the regularization parameter. We show that, as long as the RKHS is infinite-dimensional, there is a threshold on r, above which, the spectrally-truncated KRR surprisingly outperforms the full KRR in terms of the minimax risk, where the minimum is taken over the regularization parameter. This strengthens the existing results on approximation schemes, by showing that not only one does not lose in terms of the rates, truncation can in fact improve the performance, for all finite samples (above the threshold). Moreover, we show that the implicit regularization achieved by spectral truncation is not a substitute for Hilbert norm regularization. Both are needed to achieve the best performance.

In the common time series model $X_{i,n}=\mathrm{\mu }(i∕n)+{\mathit{\epsilon }}_{i,n}$ with non-stationary errors we consider the problem of detecting a significant deviation of the mean function μ from a benchmark $g(\mathrm{\mu })$ (such as the initial value $\mathrm{\mu }(0)$ or the average trend ${\int }_0^1\mathrm{\mu }(t)dt$). The problem is motivated by a more realistic modelling of change point analysis, where one is interested in identifying relevant deviations in a smoothly varying sequence of means ${(\mathrm{\mu }(i∕n))}_{i=1,\dots ,n}$ and cannot assume that the sequence is piecewise constant. A test for this type of hypotheses is developed using an appropriate estimator for the integrated squared deviation of the mean function and the threshold. By a new concept of self-normalization adapted to non-stationary processes an asymptotically pivotal test for the hypothesis of a relevant deviation is constructed. The results are illustrated by means of a simulation study and a data example.

This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The formula is neat and holds for deterministic matrix perturbations. Second, we calculate the expected projection distance between the empirical singular subspaces and true singular subspaces. Our method allows obtaining arbitrary k-th order approximation of the expected projection distance. Third, we prove the non-asymptotical normal approximation of the projection distance with different levels of bias corrections. By the $\lceil log(d_1+d_2)\rceil$-th order bias corrections, the asymptotical normality holds under optimal signal-to-noise ratio (SNR) condition where $d_1$ and $d_2$ denote the matrix sizes. In addition, it shows that higher order approximations are unnecessary when $|d_1-d_2|=O({(d_1+d_2)}^{1∕2})$. Finally, we provide comprehensive simulation results to merit our theoretic discoveries.

Unlike the existing results, our approach is non-asymptotical and the convergence rates are established. Our method allows the rank r to diverge as fast as $o({(d_1+d_2)}^{1∕3})$. Moreover, our method requires no eigen-gap condition (except the SNR) and no constraints between $d_1$ and $d_2$.

We introduce noncausal counting processes, defined by time-reversing an INAR(1) process, a non-INAR(1) Markov affine counting process, or a random coefficient INAR(1) [RCINAR(1)] process. The noncausal processes are shown to be generically time irreversible and their calendar time dynamic properties are unreplicable by existing causal models. In particular, they allow for locally bubble-like explosion, while at the same time preserving stationarity. Many of these processes have also closed form calendar time conditional predictive distribution, and allow for a simple queuing interpretation, similar as their causal counterparts.

In this paper, a general stochastic optimization procedure is studied, unifying several variants of the stochastic gradient descent such as, among others, the stochastic heavy ball method, the Stochastic Nesterov Accelerated Gradient algorithm (S-NAG), and the widely used Adam algorithm. The algorithm is seen as a noisy Euler discretization of a non-autonomous ordinary differential equation, recently introduced by Belotto da Silva and Gazeau, which is analyzed in depth. Assuming that the objective function is non-convex and differentiable, the stability and the almost sure convergence of the iterates to the set of critical points are established. A noteworthy special case is the convergence proof of S-NAG in a non-convex setting. Under some assumptions, the convergence rate is provided under the form of a Central Limit Theorem. Finally, the non-convergence of the algorithm to undesired critical points, such as local maxima or saddle points, is established. Here, the main ingredient is a new avoidance of traps result for non-autonomous settings, which is of independent interest.

In this correction, we provide an acknowledgment for the source of several images in the paper: J. Hendricks, C. Neumann, C.P. Saunders (2021) “Quantification of the weight of fingerprint evidence using a ROC-based Approximate Bayesian Computation algorithm for model selection”, Electronic Journal of Statistics (15)1, pp. 1228–1262.

Incorporating information about the target distribution in proposal mechanisms generally produces efficient Markov chain Monte Carlo algorithms (or at least, algorithms that are more efficient than uninformed counterparts). For instance, it has proved successful to incorporate gradient information in fixed-dimensional algorithms, as seen with algorithms such as Hamiltonian Monte Carlo. In trans-dimensional algorithms, Green (2003) recommended to sample the parameter proposals during model switches from normal distributions with informative means and covariance matrices. These proposal distributions can be viewed as asymptotic approximations to the parameter distributions, where the limit is with regard to the sample size. Models are typically proposed using uninformed uniform distributions. In this paper, we build on the approach of Zanella (2020) for discrete spaces to incorporate information about neighbouring models. We rely on approximations to posterior model probabilities that are asymptotically exact. We prove that, in some scenarios, the samplers combining this approach with that of Green (2003) behave like ideal ones that use the exact model probabilities and sample from the correct parameter distributions, in the large-sample regime. We show that the implementation of the proposed samplers is straightforward in some cases. The methodology is applied to a real-data example. The code is available online.

This article presents a novel method for prediction of stationary functional time series, in particular for trajectories that share a similar pattern but display variable phases. The limitation of most of the existing prediction methodologies for functional time series is that they only consider vertical variation (amplitude, scale, or vertical shift). To overcome this limitation, we develop a shape-preserving (SP) prediction method that incorporates both vertical and horizontal variation. One major advantage of our proposed method is the ability to preserve the shape of functions. Moreover, our proposed SP method does not involve unnatural transformations and can be easily implemented using existing software packages. The utility of the SP method is demonstrated in the analysis of non-metanic hydrocarbons (NMHC) concentration. The analysis demonstrates that the prediction by the SP method captures the common pattern better than the existing prediction methods and also provides competitive prediction accuracy.

This paper derives a new randomization procedure by conditioning Efron’s (1971) biased coin design to a prespecified final balance. The new procedure remains a function of the original bias parameter which now controls the probability of intermediate balance rather than final balance. As the sample size increases, the design’s selection bias and intermediate balance are similar to those of the original biased coin, but unlike the original biased coin it always guarantees final balance. It is also shown that the permuted block design for equal allocation is a special case of the new procedure when used in blocks. The latter can substitute the permuted blocks with the added benefit of reducing the expected number of deterministic assignments. The new design is also noteworthy since it shows that a randomization procedure with new properties can be obtained by conditioning an existing one to a subset in its allocation space. New relationships among existing designs can be established in the process, further elucidating the protean nature of randomization.

Many studies have been conducted on flows of probability measures, often in terms of gradient flows. We utilize a generalized notion of derivatives with respect to time to model the instantaneous evolution of empirically observed one-dimensional distributions that vary over time and develop consistent estimates for these derivatives. Employing local Fréchet regression and working in local tangent spaces with regard to the Wasserstein metric, we derive the rate of convergence of the proposed estimators. The resulting time dynamics are illustrated with time-varying distribution data that include yearly income distributions and the evolution of mortality over calendar years.

We investigate a connection between spatial statistics and statistical physics to obtain new covariance functions with direct physical interpretation for spatial random fields. These covariance functions are based on the exponential Boltzmann-Gibbs representation and use an energy functional to represent interactions between the values of the random field at different points in space. This formulation results in closed-form generalized covariance functions, which display infinite variance in Euclidean spaces of dimension larger than one. We propose regularization schemes in real and reciprocal (spectral) space that lead to well-behaved covariance structures. The real-space regularization parameter allows a continuous interpolation between the Boltzmann-Gibbs covariance and the exponential covariance. We also propose discretized approximations on regular grids, and we show that they represent reparametrized versions of the well-known Besag and Leroux lattice models. We then discuss parameter estimation and spatial prediction for the regularized Boltzmann-Gibbs covariance model in two dimensions. We recommend using the pairwise difference likelihood that combines satisfactory estimation performance and good scalability with many observation points. The predictive performance of the regularized covariance function is assessed by means of cross-validation statistics. Irregularly-spaced samples from the Walker Lake dataset are used, and spatial prediction is conducted by means of ordinary kriging. The regularized Boltzmann-Gibbs covariance yields improved predictive performance compared to the exponential covariance model.

The efficiency of a Markov sampler based on the underdamped Langevin diffusion is studied for high dimensional targets with convex and smooth potentials. We consider a classical second-order integrator which requires only one gradient computation per iteration. Contrary to previous works on similar samplers, a dimension-free contraction of Wasserstein distances and convergence rate for the total variance distance are proven for the discrete time chain itself. Non-asymptotic Wasserstein and total variation efficiency bounds and concentration inequalities are obtained for both the Metropolis adjusted and unadjusted chains. In particular, for the unadjusted chain, in terms of the dimension d and the desired accuracy ε, the Wasserstein efficiency bounds are of order $\sqrt{d}∕\mathit{\epsilon }$ in the general case, $\sqrt{d∕\mathit{\epsilon }}$ if the Hessian of the potential is Lipschitz, and $d^{1∕4}∕\sqrt{\mathit{\epsilon }}$ in the case of a separable target, in accordance with known results for other kinetic Langevin or HMC schemes.

In this work, we incorporate matrix projections into the reduced rank regression method, and then develop reduced rank regression estimators based on random projection and orthogonal projection in high-dimensional multivariate linear regression model. We propose a consistent estimator of the rank of the coefficient matrix and achieve prediction performance bounds for the proposed estimators based on mean squared errors. Finally, some simulation studies and a real data analysis are carried out to demonstrate that the proposed methods possess good stability, prediction performance and rank consistency compared to some other existing methods.

This manuscript presents an approach to perform generalized linear regression with multiple high dimensional covariance matrices as the outcome. In many areas of study, such as resting-state functional magnetic resonance imaging (fMRI) studies, this type of regression can be utilized to characterize variation in the covariance matrices across units. Model parameters are estimated by maximizing a likelihood formulation of a generalized linear model, conditioning on a well-conditioned linear shrinkage estimator for multiple covariance matrices, where the shrinkage coefficients are proposed to be shared across matrices. Theoretical studies demonstrate that the proposed covariance matrix estimator is optimal achieving the uniformly minimum quadratic loss asymptotically among all linear combinations of the identity matrix and the sample covariance matrix. Under certain regularity conditions, the proposed estimator of the model parameters is consistent. The superior performance of the proposed approach over existing methods is illustrated through simulation studies. Implemented to a resting-state fMRI study acquired from the Alzheimer’s Disease Neuroimaging Initiative, the proposed approach identified a brain network within which functional connectivity is significantly associated with Apolipoprotein E ε4, a strong genetic marker for Alzheimer’s disease.

The Gaussian model equips strong properties that facilitate studying and interpreting graphical models. Specifically it reduces conditional independence and the study of positive association to determining partial correlations and their signs. When Gaussianity does not hold partial correlation graphs are a useful relaxation of graphical models, but it is not clear what information they contain (besides the obvious lack of linear association). We study elliptical and transelliptical distributions as middle-ground between the Gaussian and other families that are more flexible but either do not embed strong properties or do not lead to simple interpretation. We characterize the meaning of zero partial correlations in elliptical and elliptical copula models and show that they retain much of the dependence structure from the Gaussian case. Regarding positive dependence, we prove impossibility results to learn certain positive (trans)elliptical graphical models, including that an elliptical distribution that is multivariate totally positive of order two for all dimensions must be essentially Gaussian. We then show how to interpret positive partial correlations as a relaxation, and obtain important properties related to faithfulness and Simpson’s paradox. We illustrate the transelliptical model potential to study tail dependence in S&P500 data, and of positivity to improve regularized inference.

We propose a difference-based nonparametric methodology for the estimation and inference of the time-varying auto-covariance functions of a locally stationary time series when it is contaminated by a complex trend with both abrupt and smooth changes. Simultaneous confidence bands (SCB) with asymptotically correct coverage probabilities are constructed for the auto-covariance functions under complex trend. A simulation-assisted bootstrapping method is proposed for the practical construction of the SCB. Detailed simulation and a real data example round out our presentation.

Suppose X is a multivariate diffusion process that is observed discretely in time. At each observation time, a transformation of the state of the process is observed with noise. The smoothing problem consists of recovering the path of the process, consistent with the observations. We derive a novel Markov Chain Monte Carlo algorithm to sample from the exact smoothing distribution. The resulting algorithm is called the Backward Filtering Forward Guiding (BFFG) algorithm. We extend the algorithm to include parameter estimation. The proposed method relies on guided proposals introduced in [53]. We illustrate its efficiency in a number of challenging problems.

We investigate estimation in Poisson regression model when the count response is right-censored and the censoring indicators are missing at random. We propose several estimators based on the regression calibration, multiple imputation and augmented inverse probability weighting methods. Under appropriate regularity conditions, we prove the consistency of our estimators and we derive their asymptotic distributions. Simulation experiments are carried out to investigate the finite sample behaviour and relative performance of the proposed estimates. These estimates are illustrated on a real data set.

Hypothesis testing of structure in covariance matrices is of significant importance, but faces great challenges in high-dimensional settings. Although consistent frequentist one-sample covariance tests have been proposed, there is a lack of simple, computationally scalable, and theoretically sound Bayesian testing methods for large covariance matrices. Motivated by this gap and by the need for tests that are powerful against sparse alternatives, we propose a novel testing framework based on the maximum pairwise Bayes factor. Our initial focus is on one-sample covariance testing; the proposed test can optimally distinguish null and alternative hypotheses in a frequentist asymptotic sense. We then propose diagonal tests and a scalable covariance graph selection procedure that are shown to be consistent. A simulation study evaluates the proposed approach relative to competitors. We illustrate advantages of our graph selection method on a gene expression data set.

Envelope method was recently proposed as a method to reduce the dimension of responses in multivariate regressions. However, when there exists missing data, the envelope method using the complete case observations may lead to biased and inefficient results. In this paper, we generalize the envelope estimation when the predictors and/or the responses are missing at random. Specifically, we incorporate the envelope structure in the expectation-maximization (EM) algorithm. As the parameters under the envelope method are not pointwise identifiable, the EM algorithm for the envelope method was not straightforward and requires a special decomposition. Our method is guaranteed to be more efficient, or at least as efficient as, the standard EM algorithm. Moreover, our method has the potential to outperform the full data MLE. We give asymptotic properties of our method under both normal and non-normal cases. The efficiency gain over the standard EM is confirmed in simulation studies and in an application to the Chronic Renal Insufficiency Cohort (CRIC) study.

We study approximation theorems for the Euler characteristic of the Vietoris-Rips and Čech filtration. The filtration is obtained from a Poisson or binomial sampling scheme in the critical regime. We apply our results to the smooth bootstrap of the Euler characteristic and determine its rate of convergence in the Kantorovich-Wasserstein distance and in the Kolmogorov distance.

We consider the problem of estimating the parameters a Gaussian Mixture Model with K components of known weights, all with an identity covariance matrix. We make two contributions. First, at the population level, we present a sharper analysis of the local convergence of EM and gradient EM, compared to previous works. Assuming a separation of $\mathrm{\Omega }(\sqrt{logK})$, we prove convergence of both methods to the global optima from an initialization region larger than those of previous works. Specifically, the initial guess of each component can be as far as (almost) half its distance to the nearest Gaussian. This is essentially the largest possible contraction region. Our second contribution are improved sample size requirements for accurate estimation by EM and gradient EM. In previous works, the required number of samples had a quadratic dependence on the maximal separation between the K components, and the resulting error estimate increased linearly with this maximal separation. In this manuscript we show that both quantities depend only logarithmically on the maximal separation.

Estimation of sparse, high-dimensional precision matrices is an important and challenging problem. Existing methods all assume that observations can be made precisely but, in practice, this often is not the case; for example, the instruments used to measure the response may have limited precision. The present paper incorporates measurement error in the context of estimating a sparse, high-dimensional precision matrix. In particular, for a Gaussian graphical model with data corrupted by Gaussian measurement error with unknown variance, we establish a general result which gives sufficient conditions under which the posterior contraction rates that hold in the no-measurement-error case carry over to the measurement-error case. Interestingly, this result does not require that the measurement error variance be small. We apply our general result to several cases with well-known prior distributions for sparse precision matrices and also to a case with a newly-constructed prior for precision matrices with a sparse factor-loading form. Two different simulation studies highlight the empirical benefits of accounting for the measurement error as opposed to ignoring it, even when that measurement error is relatively small.

We investigate the mean squared error and the asymptotic normality for a class of recursive kernel estimators based on a sample of spatially dependent observations. Our main result provides sufficient conditions for a spatial version of a recursive estimator introduced by Hall and Patil (1994) to satisfy a central limit theorem. The results are stated for strongly mixing random fields in the sense of Rosenblatt (1956) and for weakly dependent random fields in the sense of Wu (2005).

We consider the problem of constructing confidence intervals for the median of a response $Y\in \mathbb{R}$ conditional on features $X\in {\mathbb{R}}^d$ in a situation where we are not willing to make any assumption whatsoever on the underlying distribution of the data $(X,Y)$. We propose a method based upon ideas from conformal prediction and establish a theoretical guarantee of coverage while also going over particular distributions where its performance is sharp. Additionally, we prove an equivalence between confidence intervals for the conditional median and confidence intervals for the response variable, resulting in a lower bound on the length of any possible conditional median confidence interval. This lower bound is independent of sample size and holds for all distributions with no point masses.

The $\mathrm{ROC}$ curve is the gold standard for measuring the performance of a test/scoring statistic regarding its capacity to discriminate between two statistical populations in a wide variety of applications, ranging from anomaly detection in signal processing to information retrieval, through medical diagnosis. Most practical performance measures used in scoring/ranking applications such as the $\mathrm{AUC}$, the local $\mathrm{AUC}$, the p-norm push, the DCG and others, can be viewed as summaries of the $\mathrm{ROC}$ curve. In this paper, the fact that most of these empirical criteria can be expressed as two-sample linear rank statistics is highlighted and concentration inequalities for collections of such random variables, referred to as two-sample rank processes here, are proved, when indexed by VC classes of scoring functions. Based on these nonasymptotic bounds, the generalization capacity of empirical maximizers of a wide class of ranking performance criteria is next investigated from a theoretical perspective. It is also supported by empirical evidence through convincing numerical experiments.

In this work, we study a new recursive stochastic algorithm for the joint estimation of quantile and superquantile of an unknown distribution. The novelty of this algorithm is to use the Cesaro averaging of the quantile estimation inside the recursive approximation of the superquantile. We provide some sharp non-asymptotic bounds on the quadratic risk of the superquantile estimator for different step size sequences. We also prove new non-asymptotic $L^p$-controls on the Robbins Monro algorithm for quantile estimation and its averaged version. Finally, we derive a central limit theorem of our joint procedure using the diffusion approximation point of view hidden behind our stochastic algorithm.

Recursive max-linear vectors model causal dependence between node variables by a structural equation model, expressing each node variable as a max-linear function of its parental nodes in a directed acyclic graph (DAG) and some exogenous innovation. For such a model, there exists a unique minimum DAG, represented by the Kleene star matrix of its edge weight matrix, which identifies the model and can be estimated. For a more realistic statistical modeling we introduce some random observational noise. A probabilistic analysis of this new noisy model reveals that the unique minimum DAG representing the distribution of the non-noisy model remains unchanged and identifiable. Moreover, the distribution of the minimum ratio estimators of the model parameters at their left limits are completely determined by the distribution of the noise variables up to a positive constant. Under a regular variation condition on the noise variables we prove that the estimated Kleene star matrix converges to a matrix of independent Weibull entries after proper centering and scaling.

In this paper, we study the nonparametric linear model, when the error process is a dependent Gaussian process. We focus on the estimation of the mean vector via a model selection approach. We first give the general theoretical form of the penalty function, ensuring that the penalized estimator among a collection of models satisfies an oracle inequality. Then we derive a penalty shape involving the spectral radius of the covariance matrix of the errors, which can be chosen proportional to the dimension when the error process is stationary and short range dependent. However, this penalty can be too rough in some cases, in particular when the error process is long range dependent. In a second part, we focus on the fixed-design regression model assuming that the error process is a stationary Gaussian process. We propose a model selection procedure in order to estimate the mean function via piecewise polynomials on a regular partition, when the error process is either short range dependent, long range dependent or anti-persistent. We present different kinds of penalties, depending on the memory of the process. For each case, an adaptive estimator is built, and the rates of convergence are computed. Thanks to several sets of simulations, we study the performance of these different penalties for all types of errors (short memory, long memory and anti-persistent errors). Finally, we give an application of our method to the well-known Nile data, which clearly shows that the type of dependence of the error process must be taken into account.

A sample covariance matrix S of completely observed data is the key statistic in a large variety of multivariate statistical procedures, such as structured covariance/precision matrix estimation, principal component analysis, and testing of equality of mean vectors. However, when the data are partially observed, the sample covariance matrix from the available data is biased and does not provide valid multivariate procedures. To correct the bias, a simple adjustment method called inverse probability weighting (IPW) has been used in previous research, yielding the IPW estimator. The estimator can play the role of S in the missing data context, thus replacing S in off-the-shelf multivariate procedures such as the graphical lasso algorithm. However, theoretical properties (e.g. concentration) of the IPW estimator have been only established in earlier work under very simple missing structures; every variable of each sample is independently subject to missingness with equal probability. We investigate the deviation of the IPW estimator when observations are partially observed under general missing dependency. We prove the optimal convergence rate $O_p(\sqrt{logp∕n})$ of the IPW estimator based on the element-wise maximum norm, even when two unrealistic assumptions (known mean and/or missing probabilities) frequently assumed to be known in the past work are relaxed. The optimal rate is especially crucial in estimating a precision matrix, because of the “meta-theorem” [26] that claims the rate of the IPW estimator governs that of the resulting precision matrix estimator. In the simulation study, we discuss one of practically important issues, non-positive semi-definiteness of the IPW estimator, and compare the estimator with imputation methods.

Finite state space hidden Markov models are flexible tools to model phenomena with complex time dependencies: any process distribution can be approximated by a hidden Markov model with enough hidden states. We consider the problem of estimating an unknown process distribution using nonparametric hidden Markov models in the misspecified setting, that is when the data-generating process may not be a hidden Markov model. We show that when the true distribution is exponentially mixing and satisfies a forgetting assumption, the maximum likelihood estimator recovers the best approximation of the true distribution. We prove a finite sample bound on the resulting error and show that it is optimal in the minimax sense–up to logarithmic factors–when the model is well specified.

An analyst is given a training set consisting of regression datasets $D_j$ of different sizes, which are distributed according to some $G_j$, $j=1,\dots ,\mathcal{J}$, where the distributions $G_j$ are assumed to form a random sample generated by some common source. In particular, the $D_j$’s have a common set of covariates and they are all labeled. The training set is used by the analyst for selection of subsets of covariates denoted by $P^{\ast }(n)$, whose role is described next.

The multi-task problem we consider is as follows: given a number of random labeled datasets (which may be in the training set or not) $D_{J_k}$ of size $n_k$, $k=1,\dots ,K$, estimate separately for each dataset the regression coefficients on the subset of covariates $P^{\ast }(n_k)$ and then predict future dependent variables given their covariates.

Naturally, a large sample size $n_k$ of $D_{J_k}$ allows a larger subset of covariates, and the dependence of the size of the selected covariate subsets on $n_k$ is needed in order to achieve good prediction and avoid overfitting. Subset selection is notoriously difficult and computationally demanding, and requires large samples; using all the regression datasets in the training set together amounts to borrowing strength toward better selection under suitable assumptions. Furthermore, using common subsets for all regressions having a given sample size standardizes and simplifies the data collection and avoids having to select and use a different subset for each prediction task. Our approach is efficient when the relevant covariates for prediction are common to the different regressions, while the models’ coefficients may vary between different regressions.

Last but not least, we propose a simple and meaningful measure, GENO, that allows comparisons of the predictive value of different subsets of covariates by comparing the sample size they require in order to achieve the same prediction error.

Gaussian process modeling is a standard tool for building emulators for computer experiments, which are usually used to study deterministic functions, for example, a solution to a given system of partial differential equations. This work investigates applying Gaussian process modeling to a deterministic function from prediction and uncertainty quantification perspectives, where the Gaussian process model is misspecified. Specifically, we consider the case where the underlying function is fixed and from a reproducing kernel Hilbert space generated by some kernel function, and the same kernel function is used in the Gaussian process modeling as the correlation function for prediction and uncertainty quantification. While upper bounds and the optimal convergence rate of prediction in the Gaussian process modeling have been extensively studied in the literature, a comprehensive exploration of convergence rates and theoretical study of uncertainty quantification is lacking. We prove that, if one uses maximum likelihood estimation to estimate the variance in Gaussian process modeling, under different choices of the regularization parameter value, the predictor is not optimal and/or the confidence interval is not reliable. In particular, lower bounds of the prediction error under different choices of the regularization parameter value are obtained. The results indicate that, if one directly applies Gaussian process modeling to a fixed function, the reliability of the confidence interval and the optimality of the predictor cannot be achieved at the same time.

We study the problem of the non-parametric estimation for the density π of the stationary distribution of the multivariate stochastic differential equation with jumps ${(X_t)}_{0\le t\le T}$, when the dimension d is such that $d\ge 3$. From the continuous observation of the sampling path on $[0,T]$ we show that, under anisotropic Hölder smoothness constraints, kernel based estimators can achieve fast convergence rates. In particular, they are as fast as the ones found by Dalalyan and Reiss [11] for the estimation of the invariant density in the case without jumps under isotropic Hölder smoothness constraints. Moreover, they are faster than the ones found by Strauch [32] for the invariant density estimation of continuous stochastic differential equations, under anisotropic Hölder smoothness constraints. Furthermore, we obtain a minimax lower bound on the L2-risk for pointwise estimation, with the same rate up to a $log(T)$ term. It implies that, on a class of diffusions whose invariant density belongs to the anisotropic Holder class we are considering, it is impossible to find an estimator with a rate of estimation faster than the one we propose.

Edge-exchangeable probabilistic network models generate edges as an i.i.d. sequence from a discrete measure, providing a simple means for statistical inference of latent network properties. The measure is often constructed using the self-product of a realization from a Bayesian nonparametric (BNP) discrete prior; but unlike in standard BNP models, the self-product measure prior is not conjugate the likelihood, hindering the development of exact simulation and inference algorithms. Approximation via finite truncation of the discrete measure is a straightforward alternative, but incurs an unknown approximation error. In this paper, we develop methods for forward simulation and posterior inference in random self-product-measure models based on truncation, and provide theoretical guarantees on the quality of the results as a function of the truncation level. The techniques we present are general and extend to the broader class of discrete Bayesian nonparametric models.

The generalized gamma convolutions class of distributions appeared in Thorin’s work while looking for the infinite divisibility of the log-Normal and Pareto distributions. Although these distributions have been extensively studied in the univariate case, the multivariate case and the dependence structures that can arise from it have received little interest in the literature. Furthermore, only one projection procedure for the univariate case was recently constructed, and no estimation procedures are available. By expanding the densities of multivariate generalized gamma convolutions into a tensorized Laguerre basis, we bridge the gap and provide performant estimation procedures for both the univariate and multivariate cases. We provide some insights about performance of these procedures, and a convergent series for the density of multivariate gamma convolutions, which is shown to be more stable than Moschopoulos’s and Mathai’s univariate series. We furthermore discuss some examples.

We observe a stochastic process Y on ${[0,1]}^d$ ($d\ge 1$) satisfying $dY(t)=n^{1∕2}f(t)dt+dW(t)$, $t\in {[0,1]}^d$, where $n\ge 1$ is a given scale parameter (‘sample size’), W is the standard Brownian sheet on ${[0,1]}^d$ and $f\in L_1({[0,1]}^d)$ is the unknown function of interest. We propose a multivariate multiscale statistic in this setting and prove that the statistic attains a subexponential tail bound; this extends the work of Dümbgen and Spokoiny [11] who proposed the analogous statistic for $d=1$. In the process, we generalize Theorem 6.1 of Dümbgen and Spokoiny [11] about stochastic processes with sub-Gaussian increments on a pseudometric space, which is of independent interest. We use the proposed multiscale statistic to construct optimal tests (in an asymptotic minimax sense) for testing $f=0$ versus (i) appropriate Hölder classes of functions, and (ii) alternatives of the form $f={\mathrm{\mu }}_n{\mathbb{I}}_{B_n}$, where $B_n$ is an axis-aligned hyperrectangle in ${[0,1]}^d$ and ${\mathrm{\mu }}_n\in \mathbb{R}$; ${\mathrm{\mu }}_n$ and $B_n$ unknown.

In functional data the interest is to find a global mean pattern, but also to capture the individual curve differences in phase and amplitude. This can be done conveniently by building in random effects on two levels: in the warping functions to account for individual phase variations; and in the linear structure to deal with individual amplitude variations. Via an appropriate choice of the warping function and B-spline approximations, estimation in the nonlinear mixed effects functional model is feasible, and does not require any prior knowledge on landmarks for the functional data. Sufficient and necessary conditions for identifiability of the flexible model are provided. A theoretical study is conducted: we establish asymptotic normality and consistency of the estimators of the registration and amplitude models, convergence of the iterative process, and consistency of the final estimator provided by the iterative process. The finite-sample performance of the proposed estimation procedure is investigated in a simulation study, which includes comparisons with existing methods. The added value of the developed method is further illustrated via the analysis of a real data example.

Medical imaging helps to detect and monitor internal irregularities in the human body. We leverage a block median filtering technique to model pixel-to-pixel differences between two images to develop automated detection of abnormalities in noisy medical images. We propose two robust detection methods, with the test statistic being the conventional maxima and the scale-invariant ratio of the medians from partitioned image grids. Theoretically, we investigate the asymptotic behaviors of two proposed tests. Numerically, we carry out simulation studies to investigate the type I error rate and the power of two tests. In addition, a real application in medical images with gastrointestinal bleeding demonstrates the outperformance and efficiency of the ratio test method. Besides, the developed tests can also be applied to problems in other scientific fields, e.g., air pollution detection using collected remote sensing hyperspectral images.

Personalized prediction is of high interest in medicine; potential applications include the prediction of individual drug responses or risks of complications. But typical statistical pipelines such as ridge estimation combined with cross-validation ignore the heterogeneity among the patients and, therefore, are not suited for personalized prediction. We, therefore, introduce an alternative ridge-type pipeline that can minimize the prediction error of each patient individually. We show that our pipeline is optimal in terms of oracle inequalities, fast, and highly effective both in simulations and on real data.

We consider nonparametric estimation of an accelerated failure-time model when the response variable is randomly censored on the right, and regressors are not mean independent of the error component. This dependence can arise, for instance, because of measurement error. We achieve identification and conduct estimation using a vector of instrumental variables. Censoring is independent of the response variable given the instruments. We consider settings in which regressors are continuously distributed. However, the instruments may or may not be continuous, and we show how various independence restrictions allow us to identify and estimate the unknown function of interest depending on the nature of instruments. We provide rates of convergence of our estimator and showcase its finite sample properties in simulations.

The asymptotic behaviour of Linear Spectral Statistics (LSS) of the smoothed periodogram estimator of the spectral coherency matrix of a complex Gaussian high-dimensional time series ${({\mathbf{y}}_n)}_{n\in \mathbb{Z}}$ with independent components is studied under the asymptotic regime where the sample size N converges towards $+\mathrm{\infty }$ while the dimension M of y and the smoothing span of the estimator grow to infinity at the same rate in such a way that $\frac{M}{N}\to 0$. It is established that, at each frequency, the estimated spectral coherency matrix is close to the sample covariance matrix of an independent identically ${\mathcal{N}}_{\mathbb{C}}(0,{\mathbf{I}}_M)$ distributed sequence, and that its empirical eigenvalue distribution converges towards the Marcenko-Pastur distribution. This allows to conclude that each LSS has a deterministic behaviour that can be evaluated explicitly. Using concentration inequalities, it is shown that the order of magnitude of the supremum over the frequencies of the deviation of each LSS from its deterministic approximation is of the order of $\frac{1}{M}+\frac{\sqrt{M}}{N}+{(\frac{M}{N})}^3$ where N is the sample size. Numerical simulations supports our results.

The paper presents new metrics to quantify and test for (i) the equality of distributions and (ii) the independence between two high-dimensional random vectors. We show that the energy distance based on the usual Euclidean distance cannot completely characterize the homogeneity of two high-dimensional distributions in the sense that it only detects the equality of means and the traces of covariance matrices in the high-dimensional setup. We propose a new class of metrics which inherits the desirable properties of the energy distance and maximum mean discrepancy/(generalized) distance covariance and the Hilbert-Schmidt Independence Criterion in the low-dimensional setting and is capable of detecting the homogeneity of/completely characterizing independence between the low-dimensional marginal distributions in the high dimensional setup. We further propose t-tests based on the new metrics to perform high-dimensional two-sample testing/independence testing and study their asymptotic behavior under both high dimension low sample size (HDLSS) and high dimension medium sample size (HDMSS) setups. The computational complexity of the t-tests only grows linearly with the dimension and thus is scalable to very high dimensional data. We demonstrate the superior power behavior of the proposed tests for homogeneity of distributions and independence via both simulated and real datasets.

We generalize Catoni’s M-estimator, put forward in [3] by Catoni under finite variance assumption, to the case in which distributions can have finite α-th moment with $\mathit{\alpha }\in (1,2)$. Our approach, inspired by the Taylor-like expansion developed in [4], is via slightly modifying the influence function φ in [3]. A deviation bound is established for this generalized estimator, and coincides with that in [3] as $\mathit{\alpha }\uparrow 2$. Experiment shows that our M-estimator performs better than the empirical mean, the smaller the α is, the better the performance will be. As an application, we study an ${\ell }_1$ regression considered by Zhang et al. [19], who assumed that samples have finite variance, under finite α-th moment assumption with $\mathit{\alpha }\in (1,2)$.

A recommender system learns to predict the user-specific preference or intention over many items simultaneously for all users, making personalized recommendations based on a relatively small number of observations. One central issue is how to leverage three-way interactions, referred to as user-item-stage dependencies on a monotonic chain of events, to enhance the prediction accuracy. A monotonic chain of events occurs, for instance, in an article sharing dataset, where a “follow” action implies a “like” action, which in turn implies a “view” action. In this article, we develop a multistage recommender system utilizing a two-level monotonic property characterizing a monotonic chain of events for personalized prediction. Particularly, we derive a large-margin classifier based on a nonnegative additive latent factor model in the presence of a high percentage of missing observations, particularly between stages, reducing the number of model parameters for personalized prediction while guaranteeing prediction consistency. On this ground, we derive a regularized cost function to learn user-specific behaviors at different stages, linking decision functions to numerical and categorical covariates to model user-item-stage interactions. Computationally, we derive an algorithm based on blockwise coordinate descent. Theoretically, we show that the two-level monotonic property enhances the accuracy of learning as compared to a standard method treating each stage individually and an ordinal method utilizing only one-level monotonicity. Finally, the proposed method compares favorably with existing methods in simulations and an article sharing dataset.

This article discusses an informative goodness-of-fit (iGOF) approach to study multivariate distributions. When the null model is rejected, iGOF allows us to identify the underlying sources of mismodeling and naturally equips practitioners with additional insights on the nature of the deviations from the true distribution. The informative character of the procedure is achieved by exploiting smooth tests and random field theory to facilitate the analysis of multivariate data. Simulation studies show that iGOF enjoys high power for different types of alternatives. The methods presented here directly address the problem of background mismodeling arising in physics and astronomy. It is in these areas that the motivation of this work is rooted.

The logistic linear mixed model (LLMM) is one of the most widely used statistical models. Generally, Markov chain Monte Carlo algorithms are used to explore the posterior densities associated with the Bayesian LLMMs. Polson, Scott and Windle’s (2013) Pólya-Gamma data augmentation (DA) technique can be used to construct full Gibbs (FG) samplers for the LLMMs. Here, we develop efficient block Gibbs (BG) samplers for Bayesian LLMMs using the Pólya-Gamma DA method. We compare the FG and BG samplers in the context of a real data example, as the correlation between the fixed effects and the random effects changes as well as when the dimensions of the design matrices vary. These numerical examples demonstrate superior performance of the BG samplers over the FG samplers. We also derive conditions guaranteeing geometric ergodicity of the BG Markov chain when the popular improper uniform prior is assigned on the regression coefficients, and proper or improper priors are placed on the variance parameters of the random effects. This theoretical result has important practical implications as it justifies the use of asymptotically valid Monte Carlo standard errors for Markov chain based estimates of the posterior quantities.

Any decision about the release of microdata for public use is supported by the estimation of measures of disclosure risk, the most popular being the number ${\mathit{\tau }}_1$ of sample uniques that are also population uniques. In such a context, parametric and nonparametric partition-based models have been shown to have: i) the strength of leading to estimators of ${\mathit{\tau }}_1$ with desirable features, including ease of implementation, computational efficiency and scalability to massive data; ii) the weakness of producing underestimates of ${\mathit{\tau }}_1$ in realistic scenarios, with the underestimation getting worse as the tail behaviour of the empirical distribution of microdata gets heavier. To fix this underestimation phenomenon, we propose a Bayesian nonparametric partition-based model that can be tuned to the tail behaviour of the empirical distribution of microdata. Our model relies on the Pitman–Yor process prior, and it leads to a novel estimator of ${\mathit{\tau }}_1$ with all the desirable features of partition-based estimators and that, in addition, allows to reduce underestimation by tuning a “discount” parameter. We show the effectiveness of our estimator through its application to synthetic data and real data.

In this paper, we consider the multi-armed bandit problem with high-dimensional features. First, we prove a minimax lower bound, $\mathcal{O}({(logd)}^{\frac{\mathit{\alpha }+1}{2}}{T}^{\frac{1-\mathit{\alpha }}{2}}+logT)$, for the cumulative regret, in terms of horizon T, dimension d and a margin parameter $\mathit{\alpha }\in [0,1]$, which controls the separation between the optimal and the sub-optimal arms. This new lower bound unifies existing regret bound results that have different dependencies on T due to the use of different values of margin parameter α explicitly implied by their assumptions. Second, we propose a simple and computationally efficient algorithm inspired by the general Upper Confidence Bound (UCB) strategy that achieves a regret upper bound matching the lower bound. The proposed algorithm uses a properly centered ${\ell }_1$-ball as the confidence set in contrast to the commonly used ellipsoid confidence set. In addition, the algorithm does not require any forced sampling step and is thereby adaptive to the practically unknown margin parameter. Simulations and a real data analysis are conducted to compare the proposed method with existing ones in the literature.

Quantile regression has been successfully used to study heterogeneous and heavy-tailed data. Varying-coefficient models are frequently used to capture changes in the effect of input variables on the response as a function of an index or time. In this work, we study high-dimensional varying-coefficient quantile regression models and develop new tools for statistical inference. We focus on development of valid confidence intervals and honest tests for nonparametric coefficients at a fixed time point and quantile, while allowing for a high-dimensional setting where the number of input variables exceeds the sample size. Performing statistical inference in this regime is challenging due to the usage of model selection techniques in estimation. Nevertheless, we can develop valid inferential tools that are applicable to a wide range of data generating processes and do not suffer from biases introduced by model selection. We performed numerical simulations to demonstrate the finite sample performance of our method, and we also illustrated the application with a real data example.

We obtain explicit p-Wasserstein distance error bounds between the distribution of the multi-parameter MLE and the multivariate normal distribution. Our general bounds are given for possibly high-dimensional, independent and identically distributed random vectors. Our general bounds are of the optimal $\mathcal{O}(n^{-1∕2})$ order. Explicit numerical constants are given when $p\in (1,2]$, and in the case $p>2$ the bounds are explicit up to a constant factor that only depends on p. We apply our general bounds to derive Wasserstein distance error bounds for the multivariate normal approximation of the MLE in several settings; these being single-parameter exponential families, the normal distribution under canonical parametrisation, and the multivariate normal distribution under non-canonical parametrisation. In addition, we provide upper bounds with respect to the bounded Wasserstein distance when the MLE is implicitly defined.

Let $(X_t)$ be solution of a one-dimensional McKean-Vlasov stochastic differential equation with classical drift term $V(\mathit{\alpha },x)$, self-stabilizing term $\mathrm{\Phi }(\mathit{\beta },x)$ and small noise amplitude ε. Our aim is to study the estimation of the unknown parameters $\mathit{\alpha },\mathit{\beta }$ from a continuous observation of $(X_t,t\in [0,T])$ under the double asymptotic framework ε tends to 0 and T tends to infinity. After centering and normalization of the process, uniform bounds for moments with respect to $t\ge 0$ and ε are derived. We then build an explicit approximate log-likelihood leading to consistent and asymptotically Gaussian estimators, under the condition that $\mathit{\epsilon }\sqrt{T}$ tends to 0, with original rates of convergence: the rate for the estimation of α is either ${\mathit{\epsilon }}^{-1}$ or $\sqrt{T}{\mathit{\epsilon }}^{-1}$, the rate for the estimation of β is $\sqrt{T}$. Moreover, the estimators are asymptotically efficient.

We are interested in recovering information on a stochastic block model from the subgraph discovered by an exploring random walk. Stochastic block models correspond to populations structured into a finite number of types, where two individuals are connected by an edge independently from the other pairs and with a probability depending on their types. We consider here the dense case where the random network can be approximated by a graphon. This problem is motivated from the study of chain-referral surveys where each interviewee provides information on her/his contacts in the social network. First, we write the likelihood of the subgraph discovered by the random walk: biases are appearing since hubs and majority types are more likely to be sampled. Even for the case where the types are observed, the maximum likelihood estimator is not explicit any more. When the types of the vertices is unobserved, we use an SAEM algorithm to maximize the likelihood. Second, we propose a different estimation strategy using new results by Athreya and Röllin. It consists in de-biasing the maximum likelihood estimator proposed in Daudin et al. and that ignores the biases.

We focus on the problem of manifold estimation: given a set of observations sampled close to some unknown submanifold M, one wants to recover information about the geometry of M. Minimax estimators which have been proposed so far all depend crucially on the a priori knowledge of parameters quantifying the underlying distribution generating the sample (such as bounds on its density), whereas those quantities will be unknown in practice. Our contribution to the matter is twofold. First, we introduce a one-parameter family of manifold estimators ${({\hat{M}}_t)}_{t\ge 0}$ based on a localized version of convex hulls, and show that for some choice of t, the corresponding estimator is minimax on the class of models of ${\mathcal{C}}^2$ manifolds introduced in [21]. Second, we propose a completely data-driven selection procedure for the parameter t, leading to a minimax adaptive manifold estimator on this class of models. This selection procedure actually allows us to recover the Hausdorff distance between the set of observations and M, and can therefore be used as a scale parameter in other settings, such as tangent space estimation.

We present a new method for robust high-dimensional linear regression when the scale parameter of the additive errors is unknown. The proposed estimator is based on a penalized Huber M-estimator, for which theoretical bounds on the estimation error have recently been proposed in high-dimensional statistics literature. However, the variance of the error term in the linear model is intricately connected to the optimal parameter used to define the shape of the Huber loss. Our main idea is to use an adaptive technique, based on Lepski’s method, to overcome the difficulties of solving a joint nonconvex optimization problem with respect to the location and scale parameters. Furthermore, by including a weight term in the definition of the M-estimator, our consistency results hold even when the covariates are heavy-tailed. We then derive asymptotic normality of a one-step estimator constructed from the penalized Huber estimator, which can be used to construct confidence regions for subsets of coordinates. The one-step estimator is shown to be semiparametrically efficient when the covariates are sub-exponential. Our results substantially generalize previous work on high-dimensional inference, derived under sub-Gaussian assumptions on both the covariate and error distributions.

We study the problem of selecting features associated with extreme values in high dimensional linear regression. Normally, in linear modeling problems, the presence of abnormal extreme values or outliers is considered an anomaly which should either be removed from the data or remedied using robust regression methods. In many situations, however, the extreme values in regression modeling are not outliers but rather the signals of interest; consider traces from spiking neurons, volatility in finance, or extreme events in climate science, for example. In this paper, we propose a new method for sparse high-dimensional linear regression for extreme values which is motivated by the Subbotin, or generalized normal distribution, which we call the extreme value linear regression model. For our method, we utilize an ${\ell }_p$ norm loss where p is an even integer greater than two; we demonstrate that this loss increases the weight on extreme values. We prove consistency and variable selection consistency for the extreme value linear regression with a Lasso penalty, which we term the Extreme Lasso, and we also analyze the theoretical impact of extreme value observations on the model parameter estimates using the concept of influence functions. Through simulation studies and a real-world data example, we show that the Extreme Lasso outperforms other methods currently used in the literature for selecting features of interest associated with extreme values in high-dimensional regression.

Let X be a random variable with unknown mean and finite variance. We present a new estimator of the mean of X that is robust with respect to the possible presence of outliers in the sample, provides tight sub-Gaussian deviation guarantees without any additional assumptions on the shape or tails of the distribution, and moreover is asymptotically efficient. This is the first estimator that provably combines all these qualities in one package. Our construction is inspired by robustness properties possessed by the self-normalized sums. Theoretical findings are supplemented by numerical simulations highlighting strong performance of the proposed estimator in comparison with previously known techniques.

This article considers estimating the smoothness parameter of a class of nonstationary Gaussian random fields on ${\mathbb{R}}^d$ using irregularly spaced data observed along a curve. The set of covariance functions includes a nonstationary version of the Matérn covariance function as well as isotropic Matérn covariance function. Smoothness estimators are constructed via higher-order quadratic variations. Under mild conditions, these estimators are shown to be strongly consistent and convergence rate upper bounds are established with respect to fixed-domain asymptotics. Simulations indicate that the proposed estimators perform well for moderate sample sizes.

The problem of endogeneity in statistics and econometrics is often handled by introducing instrumental variables (IV) which fulfill the mean independence assumption, i.e. the unobservable is mean independent of the instruments. When full independence of IV’s and the unobservable is assumed, nonparametric IV regression models and nonparametric demand models lead to nonlinear integral equations with unknown integral kernels. We prove convergence rates for the mean integrated square error of the iteratively regularized Newton method applied to these problems. Compared to related results we derive stronger convergence results that rely on weaker nonlinearity restrictions. We demonstrate in numerical simulations for a nonparametric IV regression that the method produces better results than the standard model.

Population quantiles are important parameters in many applications. Enthusiasm for the development of effective statistical inference procedures for quantiles and their functions has been high for the past decade. In this article, we study inference methods for quantiles when multiple samples from linked populations are available. The research problems we consider have a wide range of applications. For example, to study the evolution of the economic status of a country, economists monitor changes in the quantiles of annual household incomes, based on multiple survey datasets collected annually. Even with multiple samples, a routine approach would estimate the quantiles of different populations separately. Such approaches ignore the fact that these populations are linked and share some intrinsic latent structure. Recently, many researchers have advocated the use of the density ratio model (DRM) to account for this latent structure and have developed more efficient procedures based on pooled data. The nonparametric empirical likelihood (EL) is subsequently employed. Interestingly, there has been no discussion in this context of the EL-based likelihood ratio test (ELRT) for population quantiles. We explore the use of the ELRT for hypotheses concerning quantiles and confidence regions under the DRM. We show that the ELRT statistic has a chi-square limiting distribution under the null hypothesis. Simulation experiments show that the chi-square distributions approximate the finite-sample distributions well and lead to accurate tests and confidence regions. The DRM helps to improve statistical efficiency. We also give a real-data example to illustrate the efficiency of the proposed method.

This paper concerns the use of a particular class of determinantal point processes (DPP), a class of repulsive spatial point processes, for Monte Carlo integration. Let $d\ge 1$, $I\subseteq \bar{d}=\{1,\dots ,d\}$ with $\mathrm{\iota }=|I|$. Using a single set of N quadrature points $\{u_1,\dots ,u_N\}$ defined, once for all, in dimension d from the realization of a specific DPP, we investigate “minimal” assumptions on the integrand in order to obtain unbiased Monte Carlo estimates of $\mathrm{\mu }(f_I)={\int }_{{[0,1]}^{\mathrm{\iota }}}{f}_I(u)\mathrm{d}u$ for any known ι-dimensional integrable function on ${[0,1]}^{\mathrm{\iota }}$. In particular, we show that the resulting estimator has variance with order $N^{-1-(2s\wedge 1)∕d}$ when the integrand belongs to some Sobolev space with regularity $s>0$. When $s>1∕2$ (which includes a large class of non-differentiable functions), the variance is asymptotically explicit and the estimator is shown to satisfy a Central Limit Theorem.

We consider nonparametric estimation of the transition operator P of a Markov chain and its transition density p where the singular values of P are assumed to decay exponentially fast. This is for instance the case for periodised, reversible multi-dimensional diffusion processes observed in low frequency.

We investigate the performance of a spectral hard thresholded Galerkin-type estimator for P and p, discarding most of the estimated singular triplets. The construction is based on smooth basis functions such as wavelets or B-splines. We show its statistical optimality by establishing matching minimax upper and lower bounds in $L^2$-loss. Particularly, the effect of the dimensionality d of the state space on the nonparametric rate improves from $2d$ to d compared to the case without singular value decay.

The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM. These insights provide unexpected relationships between SVM and two other linear classifiers: the mean difference and the maximal data piling direction. For example, we show that in many cases SVM can be viewed as a cropped version of these classifiers. By carefully exploring these connections we show how SVM tuning behavior is affected by data characteristics including: balanced vs. unbalanced classes, low vs. high dimension, separable vs. non-separable data. These results provide further insights into tuning SVM via cross-validation by explaining observed pathological behavior and motivating improved cross-validation methodology.

We introduce some new classes of unimodal rotational invariant directional distributions, which generalize von Mises–Fisher distribution. We propose three types of distributions, one of which represents axial data. For each new type we provide formulae and short computational study of parameter estimators by the method of moments and the method of maximum likelihood. The main goal of the paper is to develop the goodness of fit test to detect that sample entries follow one of the introduced generalized von Mises–Fisher distribution based on the maximum entropy principle. We use kth nearest neighbour distances estimator of Shannon entropy and prove its $L^2$-consistency. We examine the behaviour of the test statistics, find critical values and compute power of the test on simulated samples. We apply the goodness of fit test to local fiber directions in a glass fibre reinforced composite material and detect the samples which follow axial generalized von Mises–Fisher distribution.

Data piling refers to the phenomenon that training data vectors from each class project to a single point for classification. While this interesting phenomenon has been a key to understanding many distinctive properties of high-dimensional discrimination, the theoretical underpinning of data piling is far from properly established. In this work, high-dimensional asymptotics of data piling is investigated under a spiked covariance model, which reveals its close connection to the well-known ridged linear classifier. In particular, by projecting the ridge discriminant vector onto the subspace spanned by the leading sample principal component directions and the maximal data piling vector, we show that a negatively ridged discriminant vector can asymptotically achieve data piling of independent test data, essentially yielding a perfect classification. The second data piling direction is obtained purely from training data and shown to have a maximal property. Furthermore, asymptotic perfect classification occurs only along the second data piling direction.

In this paper we consider digital images for which the pixels values are given by a sequence of independent and identically distributed variables within an observation window. We proceed to the construction of an unbiased estimator for the perimeter without border effects. The study of the first and second moments of the perimeter allows to prove auto-normalised asymptotic normality results with an explicit covariance matrix consistently estimated. Theses Central Limit Theorems permit to built a consistent and empirical accessible test statistic to test the symmetry of the marginal distribution. Finally the asymptotic perimeter behaviour in large threshold limit regime is also explored. Several numerical studies are provided to illustrate the proposed testing procedures.

The linear coefficient in a partially linear model with confounding variables can be estimated using double machine learning (DML). However, this DML estimator has a two-stage least squares (TSLS) interpretation and may produce overly wide confidence intervals. To address this issue, we propose a regularization and selection scheme, regsDML, which leads to narrower confidence intervals. It selects either the TSLS DML estimator or a regularization-only estimator depending on whose estimated variance is smaller. The regularization-only estimator is tailored to have a low mean squared error. The regsDML estimator is fully data driven. The regsDML estimator converges at the parametric rate, is asymptotically Gaussian distributed, and asymptotically equivalent to the TSLS DML estimator, but regsDML exhibits substantially better finite sample properties. The regsDML estimator uses the idea of k-class estimators, and we show how DML and k-class estimation can be combined to estimate the linear coefficient in a partially linear endogenous model. Empirical examples demonstrate our methodological and theoretical developments. Software code for our regsDML method is available in the R-package dmlalg.

This paper is concerned with a unified approach to estimating regression methods based on a certain divergence and its localisation. Some past papers have demonstrated theoretically and numerically that infusing a little localisation in the likelihood-based methods for regression and for density estimation can actually improve the resulting estimators with respect to suitably defined global risk measures. Thus a variety of local likelihood methods have been suggested. We demonstrate that similar effect can also be observed in the general framework discussed in this paper and with respect to robust estimation procedures. Localised versions of robust regression estimation procedures perform better with respect to global risk measures based on minimisation of Bregman divergence measures. An intricate relationship between regression model’s inadequacy and its robustness can be better analysed by using the local approach developed in this paper. We support our claims with a short simulation study.

This paper develops a frequency domain bootstrap method for random fields on ${\mathbb{Z}}^2$. Three frequency domain bootstrap schemes are proposed to bootstrap Fourier coefficients of observations. Then, inverse-transformations are applied to obtain resamples in the spatial domain. As a main result, we establish the invariance principle of the bootstrap samples, from which it follows that the bootstrap samples preserve the correct second-order moment structure for a large class of random fields. The frequency domain bootstrap method is simple to apply and is demonstrated to be effective in various applications including constructing confidence intervals of correlograms for linear random fields, testing for signal presence using scan statistics, and testing for spatial isotropy in Gaussian random fields. Simulation studies are conducted to illustrate the finite sample performance of the proposed method and to compare with the existing spatial block bootstrap and subsampling methods.

High-dimensional group inference is an essential part of statistical methods for analysing complex data sets, including hierarchical testing, tests of interaction, detection of heterogeneous treatment effects and inference for local heritability. Group inference in regression models can be measured with respect to a weighted quadratic functional of the regression sub-vector corresponding to the group. Asymptotically unbiased estimators of these weighted quadratic functionals are constructed and a novel procedure using these estimators for inference is proposed. We derive its asymptotic Gaussian distribution which enables the construction of asymptotically valid confidence intervals and tests which perform well in terms of length or power. The proposed test is computationally efficient even for a large group, statistically valid for any group size and achieving good power performance for testing large groups with many small regression coefficients. We apply the methodology to several interesting statistical problems and demonstrate its strength and usefulness on simulated and real data.

We study the estimation of the linear discriminant with projection pursuit, a method that is unsupervised in the sense that it does not use the class labels in the estimation. Our viewpoint is asymptotic and, as our main contribution, we derive central limit theorems for estimators based on three different projection indices, skewness, kurtosis, and their convex combination. The results show that in each case the limiting covariance matrix is proportional to that of linear discriminant analysis (LDA), a supervised estimator of the discriminant. An extensive comparative study between the asymptotic variances reveals that projection pursuit gets arbitrarily close in efficiency to LDA when the distance between the groups is large enough and their proportions are reasonably balanced. Additionally, we show that consistent unsupervised estimation of the linear discriminant can be achieved also in high-dimensional regimes where the dimension grows at a suitable rate to the sample size, for example, $p_n=o(n^{1∕3})$ is sufficient under skewness-based projection pursuit. We conclude with a real data example and a simulation study investigating the validity of the obtained asymptotic formulas for finite samples.