In this paper consistency of the Frequency Domain Bootstrap for differentiable functionals of spectral density function of a linear stationary time series is discussed. The notion of influence function in the time domain on spectral measures is introduced. Moreover, the Fréchet and Hadamard differentiability of functionals of spectral measures are defined in the time domain. Sufficient and necessary conditions for consistency of the FDB in the considered problems are provided and the second order correctness is discussed for some functionals. Finally, validity of the FDB for the empirical processes is considered. As an illustration the notions of quantile and range in the time domain are discussed. A simulation study is provided, in which performance of the FDB is analyzed.

We investigate existence and properties of discrete mixture representations $P_{\theta }=\sum _{i\in E}w_{\theta }(i)\,Q_{i}$ for a given family $P_{\theta }$, $\theta \in \Theta $, of probability measures. The noncentral chi-squared distributions provide a classical example. We obtain existence results and results about geometric and statistical aspects of the problem, the latter including loss of Fisher information, Rao-Blackwellization, asymptotic efficiency and nonparametric maximum likelihood estimation of the mixing probabilities.

We study a plug in least squares estimator for the change point parameter where change is in the mean of a high dimensional random vector under subgaussian or subexponential distributions. We obtain sufficient conditions under which this estimator possesses sufficient adaptivity against plug in estimates of mean parameters in order to yield an optimal rate of convergence $O_{p}(\xi ^{-2})$ in the integer scale. This rate is preserved while allowing high dimensionality as well as a potentially diminishing jump size $\xi $, provided $s\log (p\vee T)=o(\surd (Tl_{T}))$ or $s\log ^{3/2}(p\vee T)=o(\surd (Tl_{T}))$ in the subgaussian and subexponential cases, respectively. Here $s,p,T$ and $l_{T}$ represent a sparsity parameter, model dimension, sampling period and the separation of the change point from its parametric boundary, respectively. Moreover, since the rate of convergence is free of $s,p$ and logarithmic terms of $T$, it allows the existence of limiting distributions under high dimensional asymptotics. These distributions are then derived as the argmax of a two sided negative drift Brownian motion or a two sided negative drift random walk under vanishing and non-vanishing jump size regimes, respectively, thereby allowing inference on the change point parameter. Feasible algorithms for implementation of the proposed methodology are provided. Theoretical results are supported with monte-carlo simulations.

By borrowing strengths from the difference and ratio between two sample covariance matrices, we propose three tests for testing the equality of two high-dimensional population covariance matrices. One test is shown to be powerful against dense alternatives, and the other two tests are suitable for general cases, including dense and sparse alternatives, or the mixture of the two. Based on random matrix theory, we investigate the asymptotical properties of these three tests under the null hypothesis as the sample size and the dimension tend to infinity proportionally. Limiting behaviors of the new tests are also studied under various local alternatives. Extensive simulation studies demonstrate that the proposed methods outperform or perform equally well compared with the existing tests.

We present results on parameter estimation of the linear partially observed Gaussian system of stochastic differential equations. We propose new one-step estimators which have the same asymptotic properties as the MLE, but much more simple to calculate, the estimators are so-called “estimator-processes”. The construction of the estimators is based on the equations of Kalman-Bucy filtration and the asymptotic corresponds to the small noises in the observations and state (hidden process) equations. We give conditions which provide the consistency, asymptotic normality and asymptotic efficiency of the proposed estimators.

Sobol’ sensitivity indices allow to quantify the respective effects of random input variables and their combinations on the variance of mathematical model output. We focus on the problem of Sobol’ indices estimation via a metamodeling approach where we replace the true mathematical model with a sample-based approximation to compute sensitivity indices. We propose a new method for indices quality control and obtain asymptotic and non-asymptotic risk bounds for Sobol’ indices estimates based on a general class of metamodels. Our analysis is closely connected with the problem of nonparametric function fitting using the orthogonal system of functions in the random design setting. It considers the relation between the metamodel quality and the error of the corresponding estimator for Sobol’ indices and shows the possibility of fast convergence rates in the case of noiseless observations. The theoretical results are complemented with numerical experiments for the approximations based on multivariate Legendre and Trigonometric polynomials.

Graphex processes resolve some pathologies in traditional random graph models, notably, providing models that are both projective and allow sparsity. Most of the literature on graphex processes study them from a probabilistic point of view. Techniques for inferring the parameter of these processes – the so-called graphon – are still marginal; exceptions are a few papers considering parametric families of graphons. Nonparametric estimation remains unconsidered. In this paper, we propose estimators for a selected choice of functionals of the graphon. Our estimators originate from the subsampling theory for graphex processes, hence can be seen as a form of bootstrap procedure.

The classical asymptotic theory for parametric $M$-estimators guarantees that, in the limit of infinite sample size, the excess risk has a chi-square type distribution, even in the misspecified case. We demonstrate how self-concordance of the loss allows to characterize the critical sample size sufficient to guarantee a chi-square type in-probability bound for the excess risk. Specifically, we consider two classes of losses: (i) self-concordant losses in the classical sense of Nesterov and Nemirovski, i.e., whose third derivative is uniformly bounded with the $3/2$ power of the second derivative; (ii) pseudo self-concordant losses, for which the power is removed. These classes contain losses corresponding to several generalized linear models, including the logistic loss and pseudo-Huber losses.

Our basic result under minimal assumptions bounds the critical sample size by $O(d\cdot d_{\text{eff}})$, where $d$ the parameter dimension and $d_{\text{eff}}$ the effective dimension that accounts for model misspecification. In contrast to the existing results, we only impose local assumptions that concern the population risk minimizer $\theta _{*}$. Namely, we assume that the calibrated predictors, i.e., predictors scaled by the square root of the second derivative of the loss, is subgaussian at $\theta _{*}$. Besides, for type-ii losses we require boundedness of certain measure of curvature of the population risk at $\theta _{*}$.

Our improved result bounds the critical sample size from above as \begin{equation*}O(\max \{d_{\text{eff}},d\log d\})\end{equation*} under slightly stronger assumptions. Namely, the local assumptions must hold in the neighborhood of $\theta _{*}$ given by the Dikin ellipsoid of the population risk. Interestingly, we find that, for logistic regression with Gaussian design, there is no actual restriction of conditions: the subgaussian parameter and curvature measure remain near-constant over the Dikin ellipsoid. Finally, we extend some of these results to $\ell _{1}$-penalized estimators in high dimensions.

Bootstrap for nonparametric regression has been around for more than 30 years. Nevertheless, most results are based on assuming an additive regression model with respect to independent and identical (i.i.d.) errors. An exception is the Local Bootstrap of Shi [23] for which, however, no bootstrap consistency results are available. We attempt to remedy this here while at the same time showing bootstrap consistency for a more general class of methods that fall under the heading of Model-free Bootstrap of Politis [18].

State-of-the-art learning algorithms, such as random forests or neural networks, are often qualified as “black-boxes” because of the high number and complexity of operations involved in their prediction mechanism. This lack of interpretability is a strong limitation for applications involving critical decisions, typically the analysis of production processes in the manufacturing industry. In such critical contexts, models have to be interpretable, i.e., simple, stable, and predictive. To address this issue, we design SIRUS (Stable and Interpretable RUle Set), a new classification algorithm based on random forests, which takes the form of a short list of rules. While simple models are usually unstable with respect to data perturbation, SIRUS achieves a remarkable stability improvement over cutting-edge methods. Furthermore, SIRUS inherits a predictive accuracy close to random forests, combined with the simplicity of decision trees. These properties are assessed both from a theoretical and empirical point of view, through extensive numerical experiments based on our $\mathtt{R/C}\mathtt{++}$ software implementation $\mathtt{sirus}$ available from $\mathtt{CRAN}$.

Volatility estimation based on high-frequency data is important for accurate measurement and control of financial asset risks. A Lévy process with infinite jump activity and microstructure noise is considered one of the simplest models for financial data at high-frequency. Utilizing this model, we propose a “purposely misspecified” posterior of the volatility obtained by ignoring the the process’ jump-component. The misspecified posterior is further corrected by a simple estimate of the location shift and re-scaling of the log likelihood. Our main result establishes a Bernstein-von Mises (BvM) theorem, which states that the proposed adjusted posterior is asymptotically Gaussian, centered at a consistent estimator, and with variance equal to the inverse of the Fisher information. In the absence of microstructure noise, our approach can be extended to make inferences for the integrated variance of general Itô semimartingales. Simulations are provided to demonstrate the accuracy of the resulting credible intervals, and the frequentist properties of the approximate Bayesian inference based on the adjusted posterior.

We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspond to desired subspaces or directions. This is relevant and important for problems where the behavior of data along a lower-dimensional space is of specific interest, such as dimension reduction or structured regression problems. We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix. The presented applications include direction-sensitive eigenspace perturbation bounds, relative bounds for the smallest eigenvalue, and the estimation of the single-index model. For the latter, a new estimator, derived from the analysis, with strong theoretical guarantees and superior numerical performance is proposed.

The multi-index model is a simple yet powerful high-dimensional regression model which circumvents the curse of dimensionality assuming $\mathbb{E}[Y|X]=g(A^{\top }X)$ for some unknown index space $A$ and link function $g$. In this paper we introduce a method for the estimation of the index space, and study the propagation error of an index space estimate in the regression of the link function. The proposed method approximates the index space by the span of linear regression slope coefficients computed over level sets of the data. Being based on ordinary least squares, our approach is easy to implement and computationally efficient. We prove a tight concentration bound that shows $N^{-1/2}$-convergence, but also faithfully describes the dependence on the chosen partition of level sets, hence providing guidance on the hyperparameter tuning. The estimator’s competitiveness is confirmed by extensive comparisons with state-of-the-art methods, both on synthetic and real data sets. As a second contribution, we establish minimax optimal generalization bounds for k-nearest neighbors and piecewise polynomial regression when trained on samples projected onto any $N^{-1/2}$-consistent estimate of the index space, thus providing complete and provable estimation of the multi-index model.

We consider group testing studies where a relatively inexpensive but imperfect assay and a perfectly accurate but higher-priced assay are both available. The primary goal is to accurately estimate the prevalence of a trait of interest, with the error rates of the imperfect assay treated as nuisance parameters. Considering the costs for performing the two assays and enrolling subjects, we propose a $D_{s}$-optimal mixed design to provide maximal information about the prevalence. We show that extreme values for the cost of the perfect assay lead to designs in which only one of the two assays is used, but otherwise the optimal designs use both assays. We provide a guaranteed algorithm to efficiently build an optimal design on discrete design spaces. Our computational results also show the robustness of the proposed design.

Quantification of uncertainty of a technical system is often based on a surrogate model of a corresponding simulation model. In any application the simulation model will not describe the reality perfectly, and consequently also the surrogate model will be imperfect. In this article we show how observed data of the real technical system can be used to improve such a surrogate model, and we analyze the rate of convergence of density estimates based on the improved surrogate model. The results are illustrated by applying the estimates to simulated and real data.

We study the convergence properties of a collapsed Gibbs sampler for Bayesian vector autoregressions with predictors, or exogenous variables. The Markov chain generated by our algorithm is shown to be geometrically ergodic regardless of whether the number of observations in the underlying vector autoregression is small or large in comparison to the order and dimension of it. In a convergence complexity analysis, we also give conditions for when the geometric ergodicity is asymptotically stable as the number of observations tends to infinity. Specifically, the geometric convergence rate is shown to be bounded away from unity asymptotically, either almost surely or with probability tending to one, depending on what is assumed about the data generating process. This result is one of the first of its kind for practically relevant Markov chain Monte Carlo algorithms. Our convergence results hold under close to arbitrary model misspecification.

We are given the adjacency matrix of a geometric graph and the task of recovering the latent positions. We study one of the most popular approaches which consists in using the graph distances and derive error bounds under various assumptions on the link function. In the simplest case where the link function is proportional to an indicator function, the bound matches an information lower bound that we derive.

In this article, we consider a class of regularized regression under the additive hazards model with censored survival data and propose a novel approach to achieve simultaneous group selection, variable selection, and parameter estimation for high-dimensional censored data, by combining the composite penalty and the pseudoscore. We develop a local coordinate descent (LCD) algorithm for efficient computation and subsequently establish the theoretical properties for the proposed selection methods. As a result, the selectors possess both group selection oracle property and variable selection oracle property, and thus enable us to simultaneously identify important groups and important variables within selected groups with high probability. Simulation studies demonstrate that the proposed method and LCD algorithm perform well. A real data example is provided for illustration.

The aim of sequential change-point detection is to issue an alarm when it is thought that certain probabilistic properties of the monitored observations have changed. This work is concerned with nonparametric, closed-end testing procedures based on differences of empirical distribution functions that are designed to be particularly sensitive to changes in the contemporary distribution of multivariate time series. The proposed detectors are adaptations of statistics used in a posteriori (offline) change-point testing and involve a weighting allowing to give more importance to recent observations. The resulting sequential change-point detection procedures are carried out by comparing the detectors to threshold functions estimated through resampling such that the probability of false alarm remains approximately constant over the monitoring period. A generic result on the asymptotic validity of such a way of estimating a threshold function is stated. As a corollary, the asymptotic validity of the studied sequential tests based on empirical distribution functions is proven when these are carried out using a dependent multiplier bootstrap for multivariate time series. Large-scale Monte Carlo experiments demonstrate the good finite-sample properties of the resulting procedures. The application of the derived sequential tests is illustrated on financial data.

A stochastic search method, the so-called Adaptive Subspace (AdaSub) method, is proposed for variable selection in high-dimensional linear regression models. The method aims at finding the best model with respect to a certain model selection criterion and is based on the idea of adaptively solving low-dimensional sub-problems in order to provide a solution to the original high-dimensional problem. Any of the usual $\ell _{0}$-type model selection criteria can be used, such as Akaike’s Information Criterion (AIC), the Bayesian Information Criterion (BIC) or the Extended BIC (EBIC), with the last being particularly suitable for high-dimensional cases. The limiting properties of the new algorithm are analysed and it is shown that, under certain conditions, AdaSub converges to the best model according to the considered criterion. In a simulation study, the performance of AdaSub is investigated in comparison to alternative methods. The effectiveness of the proposed method is illustrated via various simulated datasets and a high-dimensional real data example.

In Optimal Transport (OT) on a finite metric space, one defines a distance on the probability simplex that extends the distance on the ground space. The distance is the value of a Linear Programming (LP) problem on the set of non-negative-valued 2-way tables with assigned probability functions as margins. We apply to this case the methodology of moves from Algebraic Statistics (AS) and use it to derive a Monte Carlo Markov Chain (MCMC) solution algorithm.

In the probabilistic framework of multivariate regular variation, the first order behavior of heavy-tailed random vectors above large radial thresholds is ruled by a homogeneous limit measure. For a high dimensional vector, a reasonable assumption is that the support of this measure is concentrated on a lower dimensional subspace, meaning that certain linear combinations of the components are much likelier to be large than others. Identifying this subspace and thus reducing the dimension will facilitate a refined statistical analysis. In this work we apply Principal Component Analysis (PCA) to a re-scaled version of radially thresholded observations.

Within the statistical learning framework of empirical risk minimization, our main focus is to analyze the squared reconstruction error for the exceedances over large radial thresholds. We prove that the empirical risk converges to the true risk, uniformly over all projection subspaces. As a consequence, the best projection subspace is shown to converge in probability to the optimal one, in terms of the Hausdorff distance between their intersections with the unit sphere. In addition, if the exceedances are re-scaled to the unit ball, we obtain finite sample uniform guarantees to the reconstruction error pertaining to the estimated projection subspace. Numerical experiments illustrate the capability of the proposed framework to improve estimators of extreme value parameters.

Detecting structural changes in functional data is a prominent topic in statistical literature. However not all trends in the data are important in applications, but only those of large enough influence. In this paper we address the problem of identifying relevant changes in the eigenfunctions and eigenvalues of covariance kernels of $L^2[0,1]$-valued time series. By self-normalization techniques we derive pivotal, asymptotically consistent tests for relevant changes in these characteristics of the second order structure and investigate their finite sample properties in a simulation study. The applicability of our approach is demonstrated analyzing German annual temperature data.

We investigate geodesic projections of von Mises–Fisher (vMF) distributed directional data. The vMF distribution for random directions on the $(p-1)$-dimensional unit hypersphere ${\mathbb{S}}^{p-1}\subset {\mathbb{R}}^p$ plays the role of multivariate normal distribution in directional statistics. For one-dimensional circle ${\mathbb{S}}^1$, the vMF distribution is called von Mises (vM) distribution. Projections onto geodesics are one of main ingredients of modeling and exploring directional data. We show that the projection of vMF distributed random directions onto any geodesic is approximately vM-distributed, albeit not exactly the same. In particular, the distribution of the geodesic-projected score is an infinite scale mixture of vM distributions. Approximations by vM distributions are given along various asymptotic scenarios including large and small concentrations $(\mathrm{\kappa }\to \mathrm{\infty },\mathrm{\kappa }\to 0$), high-dimensions $(p\to \mathrm{\infty })$, and two important cases of double-asymptotics $(p,\mathrm{\kappa }\to \mathrm{\infty }$, $\mathrm{\kappa }∕p\to c$ or $\mathrm{\kappa }∕\sqrt{p}\to \mathrm{\lambda }$), to support our claim: geodesic projections of the vMF are approximately vM. As one of potential applications of the result, we contemplate a projection pursuit exploration of high-dimensional directional data. We show that in a high dimensional model almost all geodesic-projections of directional data are nearly vM, thus measures of non-vM-ness are a viable candidate for projection index.

We introduce a theoretical framework of elicitability and identifiability of set-valued functionals, such as quantiles, prediction intervals, and systemic risk measures. A functional is elicitable if it is the unique minimiser of an expected scoring function, and identifiable if it is the unique zero of an expected identification function; both notions are essential for forecast ranking and validation, and M- and Z-estimation. Our framework distinguishes between exhaustive forecasts, being set-valued and aiming at correctly specifying the entire functional, and selective forecasts, content with solely specifying a single point in the correct functional. We establish a mutual exclusivity result: A set-valued functional can be either selectively elicitable or exhaustively elicitable or not elicitable at all. Notably, since quantiles are well known to be selectively elicitable, they fail to be exhaustively elicitable. We further show that the classes of prediction intervals and Vorob’ev quantiles turn out to be exhaustively elicitable, hence not selectively elicitable, but still selectively identifiable. In particular, we provide a mixture representation of elementary exhaustive scores, leading the way to Murphy diagrams. We establish that the shortest prediction interval and those specified by an endpoint or midpoint in general fail to be elicitable with respect to either notion, unless an endpoint is given via a quantile. We end with a comprehensive literature review on common practice in forecast evaluation of set-valued functionals.

We offer a general Bayes theoretic framework to derive posterior contraction rates under a hierarchical prior design: the first-step prior serves to assess the model selection uncertainty, and the second-step prior quantifies the prior belief on the strength of the signals within the model chosen from the first step. In particular, we establish non-asymptotic oracle posterior contraction rates under (i) a local Gaussianity condition on the log likelihood ratio of the statistical experiment, (ii) a local entropy condition on the dimensionality of the models, and (iii) a sufficient mass condition on the second-step prior near the best approximating signal for each model. The first-step prior can be designed generically. The posterior distribution enjoys Gaussian tail behavior and therefore the resulting posterior mean also satisfies an oracle inequality, automatically serving as an adaptive point estimator in a frequentist sense. Model mis-specification is allowed in these oracle rates.

The local Gaussianity condition serves as a unified attempt of non-asymptotic Gaussian quantification of the experiments, and can be easily verified in various experiments considered in [GvdV07a] and beyond. The general results are applied in various problems including: (i) trace regression, (ii) shape-restricted isotonic/convex regression, (iii) high-dimensional partially linear regression, (iv) covariance matrix estimation in the sparse factor model, (v) detection of non-smooth polytopal image boundary, and (vi) intensity estimation in a Poisson point process model. These new results serve either as theoretical justification of practical prior proposals in the literature, or as an illustration of the generic construction scheme of a (nearly) minimax adaptive estimator for a complicated experiment.

We study change point detection and localization for univariate data in fully nonparametric settings, in which at each time point, we acquire an independent and identically distributed sample from an unknown distribution that is piecewise constant. The magnitude of the distributional changes at the change points is quantified using the Kolmogorov–Smirnov distance. Our framework allows all the relevant parameters, namely the minimal spacing between two consecutive change points, the minimal magnitude of the changes in the Kolmogorov–Smirnov distance, and the number of sample points collected at each time point, to change with the length of the time series. We propose a novel change point detection algorithm based on the Kolmogorov–Smirnov statistic and show that it is nearly minimax rate optimal. Our theory demonstrates a phase transition in the space of model parameters. The phase transition separates parameter combinations for which consistent localization is possible from the ones for which this task is statistically infeasible. We provide extensive numerical experiments to support our theory.

Hyperparameter tuning and model selection are important steps in machine learning. Unfortunately, classical hyperparameter calibration and model selection procedures are sensitive to outliers and heavy-tailed data. In this work, we construct a selection procedure which can be seen as a robust alternative to cross-validation and is based on a median-of-means principle. Using this procedure, we also build an ensemble method which, trained with algorithms and corrupted heavy-tailed data, selects an algorithm, trains it with a large uncorrupted subsample and automatically tunes its hyperparameters. In particular, the approach can transform any procedure into a robust to outliers and to heavy-tailed data procedure while tuning automatically its hyperparameters.

The construction relies on a divide-and-conquer methodology, making this method easily scalable even on a corrupted dataset. This method is tested with the LASSO which is known to be highly sensitive to outliers.

For more than a century, fingerprints have been used with considerable success to identify criminals or verify the identity of individuals. The categorical conclusion scheme used by fingerprint examiners, and more generally the inference process followed by forensic scientists, have been heavily criticised in the scientific and legal literature. Instead, scholars have proposed to characterise the weight of forensic evidence using the Bayes factor as the key element of the inference process. In forensic science, quantifying the magnitude of support is equally as important as determining which model is supported. Unfortunately, the complexity of fingerprint patterns render likelihood-based inference impossible. In this paper, we use an Approximate Bayesian Computation (ABC) model selection algorithm to quantify the weight of fingerprint evidence. We supplement the original ABC model selection algorithm using a Receiver Operating Characteristic curve to mitigate its known shortcomings with respect to the choice of a suitable threshold and the curse of dimensionality. In a sense, we offer an alternative to other methods that have tried to address the same issues. Our method is straightforward to implement, computationally efficient, and visually intuitive for lay individuals (i.e., jurors). We apply our method to quantify the weight of fingerprint evidence in forensic science, but we note that it can be applied to any other forensic pattern evidence.

In this paper, we consider a framework adapting the notion of cointegration when two asset prices are generated by a driftless Itô-semimartingale featuring jumps with infinite activity, observed regularly and synchronously at high frequency. We develop a regression based estimation of the cointegrated relations method and show the related consistency and central limit theory when there is cointegration within that framework. We also provide a Dickey-Fuller type residual based test for the null of no cointegration against the alternative of cointegration, along with its limit theory. Under no cointegration, the asymptotic limit is the same as that of the original Dickey-Fuller residual based test, so that critical values can be easily tabulated in the same way. Finite sample indicates adequate size and good power properties in a variety of realistic configurations, outperforming original Dickey-Fuller and Phillips-Perron type residual based tests, whose sizes are distorted by non ergodic time-varying variance and power is altered by price jumps. Two empirical examples consolidate the Monte-Carlo evidence that the adapted tests can be rejected while the original tests are not, and vice versa.

Goodness-of-fit tests based on the empirical Wasserstein distance are proposed for simple and composite null hypotheses involving general multivariate distributions. For group families, the procedure is to be implemented after preliminary reduction of the data via invariance. This property allows for calculation of exact critical values and p-values at finite sample sizes. Applications include testing for location–scale families and testing for families arising from affine transformations, such as elliptical distributions with given standard radial density and unspecified location vector and scatter matrix. A novel test for multivariate normality with unspecified mean vector and covariance matrix arises as a special case. For more general parametric families, we propose a parametric bootstrap procedure to calculate critical values. The lack of asymptotic distribution theory for the empirical Wasserstein distance means that the validity of the parametric bootstrap under the null hypothesis remains a conjecture. Nevertheless, we show that the test is consistent against fixed alternatives. To this end, we prove a uniform law of large numbers for the empirical distribution in Wasserstein distance, where the uniformity is over any class of underlying distributions satisfying a uniform integrability condition but no additional moment assumptions. The calculation of test statistics boils down to solving the well-studied semi-discrete optimal transport problem. Extensive numerical experiments demonstrate the practical feasibility and the excellent performance of the proposed tests for the Wasserstein distance of order $p=1$ and $p=2$ and for dimensions at least up to $d=5$. The simulations also lend support to the conjecture of the asymptotic validity of the parametric bootstrap.

Current status data arise commonly in applications when there is only one feasible observation time to check if the failure time has occurred, but the exact failure time remains unknown. To accommodate the covariate effect on failure time, the accelerated failure time (AFT) model has been widely used to analyze current status data with the distribution of the failure time assumed to be specified or unspecified. In this paper, we consider a logistic regression with a misclassfied covariate from the current status observation scheme. A semiparametric AFT model was built to model current status data to eliminate the bias caused by this misclassification. This model is also robust to the misspecification of the failure time compared to the parametric AFT model, as we assume an unknown distribution of the failure time in the proposed model. Furthermore, incorporating the covariate effect on the failure time increases the flexibility of the model. Finally, we adapt the Expectation-Maximization algorithm for estimation, which guarantees the convergence of the estimate. Both theory and empirical studies show the consistency of the estimator.

In this paper, we develop an asymptotic ${\mathrm{\chi }}^2$ test for detecting differences among mean functions when sparse and irregular observations are drawn from the underlying continuous stochastic processes for each subject. We provide theoretical arguments to justify the effectiveness of the proposed test procedure. Numerical experiments, including simulation studies and applications to a CD4 count data set and an eBay online auction data set, are presented to demonstrate the good performance of the developed test.

We discuss parametric estimation of a degenerate diffusion system from time-discrete observations. The first component of the degenerate diffusion system has a parameter ${\mathrm{𝜃}}_1$ in a non-degenerate diffusion coefficient and a parameter ${\mathrm{𝜃}}_2$ in the drift term. The second component has a drift term parameterized by ${\mathrm{𝜃}}_3$ and no diffusion term. Asymptotic normality is proved in two different situations for an adaptive estimator for ${\mathrm{𝜃}}_3$ with some initial estimators for $({\mathrm{𝜃}}_1,{\mathrm{𝜃}}_2)$, and an adaptive one-step estimator for $({\mathrm{𝜃}}_1,{\mathrm{𝜃}}_2,{\mathrm{𝜃}}_3)$ with some initial estimators for them. Our estimators incorporate information of the increments of both components. Thanks to this construction, the asymptotic variance of the estimators for ${\mathrm{𝜃}}_1$ is smaller than the standard one based only on the first component. The convergence of the estimators for ${\mathrm{𝜃}}_3$ is much faster than the other parameters. The resulting asymptotic variance is smaller than that of an estimator only using the increments of the second component.

In this paper we consider a fully nonparametric additive regression model for responses and predictors of various natures. This includes the case of Hilbertian and incomplete (like censored or missing) responses, and continuous, nominal discrete and ordinal discrete predictors. We propose a backfitting technique that estimates this additive model, and establish the existence of the estimator and the convergence of the associated backfitting algorithm under minimal conditions. We also develop a general asymptotic theory for the estimator such as the rates of convergence and asymptotic distribution. We verify the practical performance of the proposed estimator in a simulation study. We also apply the method to various real data sets, including those for a density-valued response regressed on a mixture of continuous and nominal discrete predictors, for a compositional response regressed on a mixture of continuous and ordinal discrete predictors, and for a censored scalar response regressed on a mixture of continuous and nominal discrete predictors.

In this paper, we consider high-dimensional Gaussian graphical models where the true underlying graph is decomposable. A hierarchical G-Wishart prior is proposed to conduct a Bayesian inference for the precision matrix and its graph structure. Although the posterior asymptotics using the G-Wishart prior has received increasing attention in recent years, most of the results assume moderate high-dimensional settings, where the number of variables p is smaller than the sample size n. However, this assumption might not hold in many real applications such as genomics, speech recognition and climatology. Motivated by this gap, we investigate asymptotic properties of posteriors under the high-dimensional setting where p can be much larger than n. The pairwise Bayes factor consistency, posterior ratio consistency and graph selection consistency are obtained in this high-dimensional setting. Furthermore, the posterior convergence rate for precision matrices under the matrix ${\ell }_1$-norm is derived, which is faster than posterior convergence rates obtained in existing literature. A simulation study confirms that the proposed Bayesian procedure outperforms competitors.

In this paper, we extend the framework of the convergence of stochastic approximations. Such a procedure is used in many methods such as parameters estimation inside a Metropolis Hastings algorithm, stochastic gradient descent or stochastic Expectation Maximization algorithm. It is given by

$${𝜃}_{n+1}={𝜃}_n+{\mathrm{\Delta }}_{n+1}{H}_{{𝜃}_n}(X_{n+1}),$$

where ${(X_n)}_{n\in \mathbb{N}}$ is a sequence of random variables following a parametric distribution which depends on ${({𝜃}_n)}_{n\in \mathbb{N}}$, and ${({\mathrm{\Delta }}_n)}_{n\in \mathbb{N}}$ is a step sequence. The convergence of such a stochastic approximation has already been proved under an assumption of geometric ergodicity of the Markov dynamic. However, in many practical situations this hypothesis is not satisfied, for instance for any heavy tail target distribution in a Monte Carlo Metropolis Hastings algorithm. In this paper, we relax this hypothesis and prove the convergence of the stochastic approximation by only assuming a subgeometric ergodicity of the Markov dynamic. This result opens up the possibility to derive more generic algorithms with proven convergence. As an example, we first study an adaptive Markov Chain Monte Carlo algorithm where the proposal distribution is adapted by learning the variance of a heavy tail target distribution. We then apply our work to the Independent Component Analysis when a positive heavy tail noise leads to a subgeometric dynamic in an Expectation Maximization algorithm.

This paper proposes a novel method to recover the sparse structure of the conditional distribution, which plays a crucial role in subsequent statistical analysis such as prediction, forecasting, conditional distribution estimation and others. Unlike most existing methods that often require explicit model assumption or suffer from computational burden, the proposed method shows great advantage by making use of some desirable properties of reproducing kernel Hilbert space (RKHS). It can be efficiently implemented by optimizing its dual form and is particularly attractive in dealing with large-scale dataset. The asymptotic consistencies of the proposed method are established under mild conditions. Its effectiveness is also supported by a variety of simulated examples and a real-life supermarket dataset from Northern China.

We consider the problem of predictive density estimation under Kullback-Leibler loss in a high-dimensional Gaussian model with exact sparsity constraints on the location parameters. For non-asymptotic sparsity levels, the least favorable prior is discrete. Here, we study the first order asymptotic minimax risk of Bayes predictive density estimates where the proportion of non-zero coordinates converges to zero as dimension increases. Motivated by an optimal thresholding rule in Mukherjee and Johnstone (2015), we propose a discrete prior and show that its Bayes predictive density estimate is minimax optimal. This produces a nonsubjective discrete prior distribution that minimizes the maximum posterior predictive relative entropy regret. We discuss the decision theoretic implications and the structural differences between our proposed prior and its closest predecessor – the geometrically decaying discrete prior of Johnstone (1994a) that produced minimax optimal point estimators under quadratic loss. Through numerical experiments, we present non-asymptotic worst-case risk of our proposed estimator across different sparsity levels.

In some applications, an experimental unit is composed of two distinct but related subunits. The response from such a unit is $(X_1,X_2)$ but we observe only $Y_1=min\{X_1,X_2\}$ and $Y_2=max\{X_1,X_2\}$, i.e., the subunit identities are not observed. We call $(Y_1,Y_2)$ unordered paired observations. Based on unordered paired observations ${\{(Y_{1i},Y_{2i})\}}_{i=1}^n$, we are interested in whether the marginal distributions for $X_1$ and $X_2$ are identical. Testing methods are available in the literature under the assumptions that $\text{Var}(X_1)=\text{Var}(X_2)$ and $\text{COV}(X_1,X_2)=0$. However, by extensive simulation studies, we observe that when one or both assumptions are violated, these methods have inflated type I errors or much lower powers. In this paper, we study the likelihood ratio test statistics for various scenarios and explore their limiting distributions without these restrictive assumptions. Furthermore, we develop Bartlett correction formulae for these statistics to enhance their precision when the sample size is not large. Simulation studies and real-data examples are used to illustrate the efficacy of the proposed methods.

We consider post-selection inference for high-dimensional (generalized) linear models. Data carving from Fithian, Sun and Taylor [10] is a promising technique to perform this task. However, it suffers from the instability of the model selector and hence, may lead to poor replicability, especially in high-dimensional settings. We propose the multicarve method inspired by multisplitting to improve upon stability and replicability. Furthermore, we extend existing concepts to group inference and illustrate the applicability of the methodology also for generalized linear models.

We introduce a procedure to generate an estimator of the regression function based on a data-dependent quasi-covering of the feature space. A quasi-partition is generated from the quasi-covering and the estimator predicts the conditional empirical expectation over the cells of the quasi-partition. We provide sufficient conditions to ensure the consistency of the estimator. Each element of the quasi-covering is labeled as significant or insignificant. We avoid the condition of cell shrinkage commonly found in the literature for data-dependent partitioning estimators. This reduces the number of elements in the quasi-covering. An important feature of our estimator is that it is interpretable.

The proof of the consistency is based on a control of the convergence rate of the empirical estimation of conditional expectations, which is interesting in itself.

We consider non-parametric density estimation in the framework of local, both pure and approximate, differential privacy. In contrast to centralized privacy scenarios with a trusted curator, in the local setup anonymization must be guaranteed already on the individual data owners’ side and must therefore precede any data mining tasks. Thus, the published anonymized data should be compatible with as many statistical procedures as possible. We consider different mechanisms to establish pure and approximate differential privacy, respectively. We obtain minimax type results over Sobolev classes indexed by a smoothness parameter $s>1∕2$ for the mean squared error at a fixed point. In particular, we show that appropriately defined kernel density estimators can attain the optimal rate of convergence if the bandwidth parameter is correctly specified. Notably, the optimal convergence rate in terms of the sample size n is $n^{-(2s-1)∕(2s+1)}$ under pure differential privacy and thus deteriorated to the rate $n^{-(2s-1)∕(2s)}$ which holds both without privacy restrictions and under approximate differential privacy. Since the optimal choice of the bandwidth parameter depends on the smoothness s and is thus not accessible in practise, adaptive methods for bandwidth selection are necessary and must, in the local privacy framework, be performed based on the anonymized data only. We address this problem by means of variants of Lepski’s method tailored to the privacy setups at hand and obtain general oracle inequalities for private kernel density estimators. In the Sobolev case, the resulting adaptive estimators attain the optimal rates of convergence at least up to logarithmic factors. On the side, we discuss some critical issues related with the notion of approximate differential privacy.

We propose a statistical model for graphs with a core-periphery structure. We give a precise notion of what it means for a graph to have this structure, based on the sparsity properties of the subgraphs of core and periphery nodes. We present a class of sparse graphs with such properties, and provide methods to simulate from this class, and to perform posterior inference. We demonstrate that our model can detect core-periphery structure in simulated and real-world networks.

Deep learning has achieved notable success in various fields, including image and speech recognition. One of the factors in the successful performance of deep learning is its high feature extraction ability. In this study, we focus on the adaptivity of deep learning; consequently, we treat the variable exponent Besov space, which has a different smoothness depending on the input location x. In other words, the difficulty of the estimation is not uniform within the domain. We analyze the general approximation error of the variable exponent Besov space and the approximation and estimation errors of deep learning. We note that the improvement based on adaptivity is remarkable when the region upon which the target function has less smoothness is small and the dimension is large. Moreover, the superiority to linear estimators is shown with respect to the convergence rate of the estimation error.

In many contemporary applications such as longitudinal studies, neuroimaging or civil engineering, a dataset can contain high dimensional measurements on both matrix-valued and vector-valued variables. Such structure demands statistical tools that can extract information from both types of measurements. In this paper, we propose a double fused Lasso regularized method to handle both matrix-valued and vector-valued predictors under the context of linear regression and logistic regression. An efficient and scalable sGS-ADMM (symmetric Gauss-Seidel based alternating direction method of multipliers) algorithm is derived to obtain the estimator. Global convergence and the Q-linear rate of convergence for the algorithm is established. Consistency of the double fused Lasso estimators holds under mild conditions. Numerical experiments and examples show that the double fused Lasso estimators achieve efficient gains in estimation and better prediction performance compared to existing estimators.

Motivated by the increasing popularity and the seemingly broad applicability of pair-copula constructions underlined by numerous publications in the last decade, in this contribution we tackle the unavoidable question on how flexible and simplifying the commonly used ‘simplifying assumption’ is from an analytic perspective and provide answers to two related open questions posed by Nagler and Czado in 2016. Aiming at a simplest possible setup for deriving the main results we first focus on the three-dimensional setting. We prove that the family of simplified copulas is flexible in the sense that it is dense in the set of all three-dimensional copulas with respect to the uniform metric $d_{\mathrm{\infty }}$ – considering stronger notions of convergence like the one induced by the metric $D_1$, by weak conditional convergence, by total variation, or by Kullback-Leibler divergence, however, the family even turns out to be nowhere dense and hence insufficient for any kind of flexible approximation. Furthermore, returning to $d_{\mathrm{\infty }}$ we show that the partial vine copula is never the optimal simplified copula approximation of a given, non-simplified copula C, and derive examples illustrating that the corresponding approximation error can be strikingly large and extend to more than 28% of the diameter of the metric space. Moreover, the mapping ψ assigning each three-dimensional copula its unique partial vine copula turns out to be discontinuous with respect to $d_{\mathrm{\infty }}$ (but continuous with respect to $D_1$ and to weak conditional convergence), implying a surprising sensitivity of partial vine copula approximations. The afore-mentioned main results concerning $d_{\mathrm{\infty }}$ are then extended to the general multivariate setting.

We consider the problem of both prediction and model selection in high dimensional generalized linear models. Predictive performance can be improved by leveraging structure information among predictors. In this paper, a graphic model-based doubly sparse regularized estimator is discussed under the high dimensional generalized linear models, that utilizes the graph structure among the predictors. The graphic information among predictors is incorporated node-by-node using a decomposed representation and the sparsity is encouraged both within and between the decomposed components. We propose an efficient iterative proximal algorithm to solve the optimization problem. Statistical convergence rates and selection consistency for the doubly sparse regularized estimator are established in the ultra-high dimensional setting. Specifically, we allow the dimensionality grows exponentially with the sample size. We compare the estimator with existing methods through numerical analysis on both simulation study and a microbiome data analysis.

A new form of the discrepancy principle for Poisson inverse problems with compact operators is proposed and discussed in relation to various other proposals. It is shown that filter-induced spectral regularization with a priori chosen smoothing parameter produces estimators that are rate-minimax under source conditions on the estimated function. With the discrepancy principle used for a posteriori choice of the smoothing parameter, the filter-induced solutions are consistent (in probability), but the convergence rates under source conditions are suboptimal, at least in the finitely smoothing case, which often happens when discrepancy principles are used in stochastic inverse problems. Finite sample performance of the proposed procedure applied to a stereological problem of Spektor, Lord and Willis is illustrated with a simulation experiment.

This paper addresses the case where data come as point sets, or more generally as measures. Our goal is to build from data an embedding of these measures into a finite-dimensional Euclidean space, that allows for provably efficient clustering of the source measures.

The vectorization technique we propose relies on finding a compactly supported approximation of the mean measure generating process, that coincides with the intensity measure in the point process framework. To this aim we provide two algorithms that we prove almost minimax optimal.

We assess the practical validity of our approach, first by showing that our results apply in the framework of persistence-based shape classification via the ATOL procedure described in [34]. At last, numerical experiments are carried out on simulated and real datasets, encompassing text classification and large-scale graph classification.

In a regular full exponential family, the maximum likelihood estimator (MLE) need not exist in the traditional sense. However, the MLE may exist in the completion of the exponential family. Existing algorithms for finding the MLE in the completion solve many linear programs; they are slow in small problems and too slow for large problems. We provide new, fast, and scalable methodology for finding the MLE in the completion of the exponential family. This methodology is based on conventional maximum likelihood computations which come close, in a sense, to finding the MLE in the completion of the exponential family. These conventional computations construct a likelihood maximizing sequence of canonical parameter values which goes uphill on the likelihood function until they meet a convergence criteria. Nonexistence of the MLE in this context results from a degeneracy of the canonical statistic of the exponential family, the canonical statistic is on the boundary of its support. There is a correspondance between this boundary and the null eigenvectors of the Fisher information matrix. Convergence of Fisher information along a likelihood maximizing sequence follows from cumulant generating function (CGF) convergence along a likelihood maximizing sequence, conditions for which are given. This allows for the construction of necessarily one-sided confidence intervals for mean value parameters when the MLE exists in the completion. We demonstrate our methodology on three examples in the main text and three additional examples in an accompanying technical report. We show that when the MLE exists in the completion of the exponential family, our methodology provides statistical inference that is much faster than existing techniques.

Principal component regression is an effective dimension reduction method for regression problems. To apply it in practice, one typically starts by selecting the number of principal components k, then estimates the corresponding regression parameters using say maximum likelihood, and finally obtains predictions with the fitted results. The success of this approach highly depends on the choice of k, and very often, due to the noisy nature of the data, it could be risky to just use one single value of k. Using the generalized fiducial inference framework, this paper develops a method for constructing a probability function on k, which provides an uncertainty measure on its value. In addition, this paper also constructs novel confidence intervals for the regression parameters and prediction intervals for future observations. The proposed methodology is backed up by theoretical results and is tested by simulation experiments and compared with other methods using real data. To the best of our knowledge, this is the first time that a full treatment for uncertainty quantification is formally considered for principal component regression.

We investigate the estimation of a density f from a n-sample on an Euclidean space ${\mathbb{R}}^D$, when the data are supported by an unknown submanifold M of possibly unknown dimension $d<D$, under a reach condition. We investigate several nonparametric kernel methods, with data-driven bandwidths that incorporate some learning of the geometry via a local dimension estimator. When f has Hölder smoothness β and M has regularity α, our estimator achieves the rate $n^{-\mathit{\alpha }\wedge \mathit{\beta }∕(2\mathit{\alpha }\wedge \mathit{\beta }+d)}$ for a pointwise loss. The rate does not depend on the ambient dimension D and we establish that our procedure is asymptotically minimax for $\mathit{\alpha }\ge \mathit{\beta }$. Following Lepski’s principle, a bandwidth selection rule is shown to achieve smoothness adaptation. We also investigate the case $\mathit{\alpha }\le \mathit{\beta }$: by estimating in some sense the underlying geometry of M, we establish in dimension $d=1$ that the minimax rate is $n^{-\mathrm{\beta }∕(2\mathrm{\beta }+1)}$ proving in particular that it does not depend on the regularity of M. Finally, a numerical implementation is conducted on some case studies in order to confirm the practical feasibility of our estimators.

We establish that Laplace transforms of the posterior Dirichlet process converge to those of the limiting Brownian bridge process in a neighbourhood about zero, uniformly over Glivenko-Cantelli function classes. For real-valued random variables and functions of bounded variation, we strengthen this result to hold for all real numbers. This last result is proved via an explicit strong approximation coupling inequality.

Current status data are commonly encountered in medical and epidemiological studies in which the failure time for study units is the outcome variable of interest. Data of this form are characterized by the fact that the failure time is not directly observed but rather is known relative to an observation time, i.e., the failure times are either left- or right-censored. Due to its structure, the analysis of such data can be challenging. To circumvent these challenges and to provide for a flexible modeling construct which can be used to analyze current status data, herein a partially linear additive transformation model is proposed. In the formulation of this model, constrained B-splines are employed to model the monotone transformation function and nonparametric covariate effects. To provide for more efficient estimators, a penalization technique is used to regularize the estimation of all unknown functions. An easy to implement hybrid algorithm is developed for model fitting, and a simple and consistent estimator of the large-sample variance-covariance matrix for regression parameter estimators is proposed. It is shown theoretically that the proposed estimators of the finite-dimensional regression coefficients are root-n consistent, asymptotically normal, and achieve the semiparametric information bound, while the estimators of the nonparametric components attain the optimal rate of convergence. The finite-sample performance of the proposed methodology is evaluated through extensive numerical studies and is further demonstrated through the analysis of human papillomavirus (HPV) data.

The aim of online monitoring is to issue an alarm as soon as there is significant evidence in the collected observations to suggest that the underlying data generating mechanism has changed. This work is concerned with open-end, nonparametric procedures that can be interpreted as statistical tests. The proposed monitoring schemes consist of computing the so-called retrospective CUSUM statistic (or minor variations thereof) after the arrival of each new observation. After proposing suitable threshold functions for the chosen detectors, the asymptotic validity of the procedures is investigated in the special case of monitoring for changes in the mean, both under the null hypothesis of stationarity and relevant alternatives. To carry out the sequential tests in practice, an approach based on an asymptotic regression model is used to estimate high quantiles of relevant limiting distributions. Monte Carlo experiments demonstrate the good finite-sample behavior of the proposed monitoring schemes and suggest that they are superior to existing competitors as long as changes do not occur at the very beginning of the monitoring. Extensions to statistics exhibiting an asymptotic mean-like behavior are briefly discussed. Finally, the application of the derived sequential change-point detection tests is succinctly illustrated on temperature anomaly data.

Inference on vertex-aligned graphs is of wide theoretical and practical importance. There are, however, few flexible and tractable statistical models for correlated graphs, and even fewer comprehensive approaches to parametric inference on data arising from such graphs. In this paper, we consider the correlated Bernoulli random graph model (allowing different Bernoulli coefficients and edge correlations for different pairs of vertices), and we introduce a new variance-reducing technique—called balancing—that can refine estimators for model parameters. Specifically, we construct a disagreement statistic and show that it is complete and sufficient; balancing can be interpreted as Rao-Blackwellization with this disagreement statistic. We show that for unbiased estimators of functions of model parameters, balancing generates uniformly minimum variance unbiased estimators (UMVUEs). However, even when unbiased estimators for model parameters do not exist—which, as we prove, is the case with both the heterogeneity correlation and the total correlation parameters—balancing is still useful, and lowers mean squared error. In particular, we demonstrate how balancing can improve the efficiency of the alignment strength estimator for the total correlation, a parameter that plays a critical role in graph matchability and graph matching runtime complexity.

We study multivariate Gaussian random fields defined over d-dimensional spheres. First, we provide a nonparametric Bayesian framework for modeling and inference on matrix-valued covariance functions. We determine the support (under the topology of uniform convergence) of the proposed random matrices, which cover the whole class of matrix-valued geodesically isotropic covariance functions on spheres. We provide a thorough inspection of the properties of the proposed model in terms of (a) first moments, (b) posterior distributions, and (c) Lipschitz continuities. We then provide an approximation method for multivariate fields on the sphere for which measures of $L^p$ accuracy are established. Our findings are supported through simulation studies that show the rate of convergence when truncating a spectral expansion of a multivariate random field at a finite order. To illustrate the modeling framework developed in this paper, we consider a bivariate spatial data set of two 2019 NCEP/NCAR Flux Reanalyses.

The formal assessment of the stochastic dominance of a random pair with respect to another one is a question of interest in the economic analysis of populations. For example, a manager may wonder if the components of a portfolio are more associated than that of another competing portfolio, in which case the former is generally considered more at risk. In this paper, a new family of copula-based concordance orderings in the spirit of increasing convex and concave orderings of random pairs is introduced as a natural extension of the well-known concordance ordering. In addition, a complete statistical methodology to test the stochastic dominance of a random pair with respect to another one according to the new concordance orderings is developed. The proposed tests are nonparametric, consistent against all alternatives, and valid under serially dependent data satisfying the α-mixing assumption. The sampling properties of the tests are investigated with the help of Monte–Carlo simulations and their usefulness is illustrated on real multivariate data.

In this paper we revisit the classical problem of nonparametric regression, but impose local differential privacy constraints. Under such constraints, the raw data $(X_1,Y_1),...,(X_n,Y_n)$, taking values in ${\mathbb{R}}^d\times \mathbb{R}$, cannot be directly observed, and all estimators are functions of the randomised output from a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and here we add Laplace distributed noise to a discretisation of the location of a feature vector $X_i$ and to the value of its response variable $Y_i$. Based on this randomised data, we design a novel estimator of the regression function, which can be viewed as a privatised version of the well-studied partitioning regression estimator. The main result is that the estimator is strongly universally consistent, and we further establish an upper bound on the rate of convergence. Our methods and analysis also give rise to a strongly universally consistent binary classification rule for locally differentially private data.

Fithian & Hastie [7] proposed a sampling scheme called local case-control (LCC) sampling that achieves stability and efficiency by utilizing a clever adjustment pertained to the logistic model. It is particularly useful for classification with large and imbalanced data. This paper proposes a more general sampling scheme based on a working principle that data points deserve higher sampling probability if they contain more information or appear “surprising” in the sense of, for example, a large error of pilot prediction or a large absolute score. Compared with the relevant existing sampling schemes, as reported in [7] and [1], the proposed one has several advantages. It adaptively gives out the optimal forms to a variety of objectives, including the LCC and [1] as special cases. Under same model specifications, the proposed estimator also performs no worse than those in the literature. The estimation procedure is valid even if the model is misspecified and/or the pilot estimator is inconsistent or dependent on full data. We present theoretical justifications of the claimed advantages and optimality of the estimation and the sampling design. Different from [1], our large sample theory are population-wise rather than data-wise. Moreover, the proposed approach can be applied to unsupervised learning studies, since it essentially only requires a specific loss function and no response-covariate structure of data is needed. Numerical studies are carried out and the evidence in support of the theory is shown.

We prove a non-asymptotic concentration inequality for the spectral norm of sparse inhomogeneous random tensors with Bernoulli entries. For an order-k inhomogeneous random tensor T with sparsity $p_{max}\ge \frac{clogn}{n}$, we show that $\Vert T-\mathbb{E}T\Vert =O(\sqrt{n{p}_{max}}{log}^{k-2}(n))$ with high probability. The optimality of this bound up to polylog factors is provided by an information theoretic lower bound. By tensor unfolding, we extend the range of sparsity to $p_{max}\ge \frac{clogn}{n^m}$ with $1\le m\le k-1$ and obtain concentration inequalities for different sparsity regimes. We also provide a simple way to regularize T such that $O(\sqrt{n^m{p}_{max}})$ concentration still holds down to sparsity $p_{max}\ge \frac{c}{n^m}$ with $k∕2\le m\le k-1$. We present our concentration and regularization results with two applications: (i) a randomized construction of hypergraphs of bounded degrees with good expander mixing properties, (ii) concentration of sparsified tensors under uniform sampling.

To quantify uncertainty around point estimates of conditional objects such as conditional means or variances, parameter uncertainty has to be taken into account. Attempts to incorporate parameter uncertainty are typically based on the unrealistic assumption of observing two independent processes, where one is used for parameter estimation, and the other for conditioning upon. Such unrealistic foundation raises the question whether these intervals are theoretically justified in a realistic setting. This paper presents an asymptotic justification for this type of intervals that does not require such an unrealistic assumption, but relies on a sample-split approach instead. By showing that our sample-split intervals coincide asymptotically with the standard intervals, we provide a novel, and realistic, justification for confidence intervals of conditional objects. The analysis is carried out for a rich class of time series models. We also present the results of a simulation study to evaluate the performance of the sample-split approach. The results indicate that also in practice sample-split intervals might be more appropriate than the standard intervals.

In a regression model, we write the Nadaraya-Watson estimator of the regression function as the quotient of two kernel estimators, and propose a bandwidth selection method for both the numerator and the denominator. We prove risk bounds for both data driven estimators and for the resulting ratio. The simulation study confirms that both estimators have good performances, compared to the ones obtained by cross-validation selection of the bandwidth. However, unexpectedly, the single-bandwidth cross-validation estimator is found to be much better than the ratio of the previous two good estimators, in the small noise context. However, the two methods have similar performances in models with large noise.

We consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of n means based on their estimates together with the (known) standard errors of those estimates. We present a generic method based on the partitioning principle in which the parameter space is partitioned into disjoint subsets and then each one of them is tested at level α. The resulting CIs have then a simultaneous coverage of $1-\mathit{\alpha }$. We show that any procedure which produces simultaneous CIs for ranks can be written as a partitioning procedure. We present a first example where we test the partitions using the likelihood ratio (LR) test. Then, in a second example we show that a recently proposed method for simultaneous CIs for ranks using Tukey’s honest significant difference test has an equivalent procedure based on the partitioning principle. By embedding these two methods inside our generic partitioning procedure, we obtain improved variants. We illustrate the performance of these methods through simulations and real data analysis on hotel ratings. While the novel method that uses the LR test and its variant produce shorter CIs when the number of means is small, the Tukey-based method and its variant produce shorter CIs when the number of means is high.

For a given function f on a multivariate domain, the level sets, given by $\{x:f(x)=c\}$ for different values of c, provide important geometrical insights about the underlying function of interest. The distance on level sets of two functions may be measured by the Hausdorff metric or a metric based on the Lebesgue measure of a discrepancy, both of which can be linked with the $L^{\mathrm{\infty }}$-distance on the underlying functions. In a Bayesian framework, we derive posterior contraction rates and optimal sized credible sets with assured frequentist coverage for level sets in some nonparametric settings by extending some univariate $L^{\mathrm{\infty }}$-posterior contraction rates to the corresponding multivariate settings. For the multivariate Gaussian white noise model, adaptive Hausdorff and Lebesgue contraction rates for levels sets of the signal function and its mixed order partial derivatives are derived using a wavelet series prior on the function. Assuming a known smoothness level of the signal function, an optimal sized credible region for a level set with assured frequentist coverage is derived based on a multidimensional trigonometric series prior. For the nonparametric regression problem, adaptive rates for level sets of the function and its mixed partial derivatives are obtained using a multidimensional wavelet series prior. When the smoothness level is given, optimal sized credible regions with assured frequentist coverage are obtained using a finite random series prior based on tensor products of B-splines. We also derive Hausdorff and Lebesgue contraction rates of a multivariate density function under a known smoothness setting.

The article describes how directional quantiles can be useful for testing the null hypothesis that a multivariate distribution is symmetric around a line in a given direction. It also generalizes the proposed tests to residual distributions in a linear regression setup, discusses their use for statistical inference regarding equality of distributions, equality of scale, or exchangeability, and illustrates the achievements with carefully designed pictures and examples.

Parameter estimation for a parabolic linear stochastic partial differential equation in one space dimension is studied observing the solution field on a discrete grid in a fixed bounded domain. Considering an infill asymptotic regime in both coordinates, we prove central limit theorems for realized quadratic variations based on temporal and spatial increments as well as on double increments in time and space. Resulting method of moments estimators for the diffusivity and the volatility parameter inherit the asymptotic normality and can be constructed robustly with respect to the sampling frequencies in time and space. Upper and lower bounds reveal that in general the optimal convergence rate for joint estimation of the parameters is slower than the usual parametric rate. The theoretical results are illustrated in a numerical example.

Cluster indices describe extremal behaviour of stationary time series. We consider their sliding blocks estimators. Using a modern theory of multivariate, regularly varying time series, we obtain central limit theorems under conditions that can be easily verified for a large class of models. In particular, we show that in the Peaks-Over-Threshold framework, sliding and disjoint blocks estimators have the same limiting variance.

Exact inference for hidden Markov models requires the evaluation of all distributions of interest – filtering, prediction, smoothing and likelihood – with a finite computational effort. This article provides sufficient conditions for exact inference for a class of hidden Markov models on general state spaces given a set of discretely collected indirect observations linked non linearly to the signal, and a set of practical algorithms for inference. The conditions we obtain are concerned with the existence of a certain type of dual process, which is an auxiliary process embedded in the time reversal of the signal, that in turn allows to represent the distributions and functions of interest as finite mixtures of elementary densities or products thereof. We describe explicitly how to update recursively the parameters involved, yielding qualitatively similar results to those obtained with Baum–Welch filters on finite state spaces. We then provide practical algorithms for implementing the recursions, as well as approximations thereof via an informed pruning of the mixtures, and we show superior performance to particle filters both in accuracy and computational efficiency. The code for optimal filtering, smoothing and parameter inference is made available in the Julia package DualOptimalFiltering.

In this article, we show that the nonparametric maximum likelihood estimator (NPMLE) of the decreasing density function in the s-sample biased sampling models, is asymptotically equivalent to a Grenander-type estimator, namely the left-continuous slope of the least concave majorant of the NPMLE of the distribution function in the larger model without imposing the monotonicity assumption. Since the two estimators favor different proof directions in establishing weak convergence, we require additional results for both estimators so that the two estimators can be considered jointly in a unified approach. For instance, we employ an analytic argument for showing the tightness of an inverse processes associated with the NPMLE, since a conventional geometric approach used in the literature cannot be employed due to multiple biased samples. We demonstrate other results using numerical simulation and a real data illustration.

Count-valued time series data are routinely collected in many application areas. We are particularly motivated to study the count time series of daily new cases, arising from the COVID-19 spread. First, we propose a Bayesian framework to study the time-varying semiparametric AR$(p)$ model for the count and then extend it to a more sophisticated time-varying INGARCH model. We calculate posterior contraction rates of the proposed Bayesian methods with respect to the average Hellinger metric. Our proposed structures of the models are amenable to Hamiltonian Monte Carlo (HMC) sampling for efficient computation. We substantiate our methods by simulations that show superiority compared to some of the existing methods that closely fit this setting. Finally, we analyze the daily time series data of newly confirmed cases in NYC to study the spread of COVID for three months.

We revisit the problem of estimating the center of symmetry θ of an unknown symmetric density f. Although Stone (1975), Van Eeden (1970), and Sacks (1975) constructed adaptive estimators of θ in this model, their estimators depend on external tuning parameters. In an effort to reduce the burden of tuning parameters, we impose an additional restriction of log-concavity on f. We construct truncated one-step estimators which are adaptive under the log-concavity assumption. Our simulations indicate that the untruncated version of the one step estimator, which is tuning parameter free, is also asymptotically efficient. We also study the maximum likelihood estimator (MLE) of θ in the shape-restricted model.

Sentiment analysis measures inclination of textual documents, aiming to extract and quantify their subjective sentiment polarity. In literature, most sentiment analysis methods first numericalize textual documents through certain word embeddings framework, and then formulate sentiment analysis as an ordinal regression or classification task. Yet it is often ignored that different people may have different preference of wording, and thus a uniform word embeddings often leads to suboptimal performance. In this article, to accommodate the heterogeneity among individual persons, we propose a covariate-assisted word embeddings in a margin-based ordinal regression framework, where covariates are incorporated through scaling factors to adjust the word embeddings. Moreover, we employ a block-wise coordinate descent scheme to tackle the resultant large-scale optimization task, and establish theoretical results to quantify the asymptotic behavior of the proposed method, guaranteeing its fast convergence rate in terms of prediction accuracy. Finally, we demonstrate the advantages of the proposed method over its competitors in both the Yelp Challenge dataset and synthetic datasets.

We study frequentist asymptotic properties of Bayesian procedures for high-dimensional Gaussian sparse regression when unknown nuisance parameters are involved. Nuisance parameters can be finite-, high-, or infinite-dimensional. A mixture of point masses at zero and continuous distributions is used for the prior distribution on sparse regression coefficients, and appropriate prior distributions are used for nuisance parameters. The optimal posterior contraction of sparse regression coefficients, hampered by the presence of nuisance parameters, is also examined and discussed. It is shown that the procedure yields strong model selection consistency. A Bernstein-von Mises-type theorem for sparse regression coefficients is also obtained for uncertainty quantification through credible sets with guaranteed frequentist coverage. Asymptotic properties of numerous examples are investigated using the theory developed in this study.

We consider estimation of the extreme value index and extreme quantiles for heavy–tailed data that are right-censored. We study a general procedure of removing low importance observations in tail estimators. This trimming procedure is applied to the state-of-the-art estimators for randomly right-censored tail estimators. Through an averaging procedure over the amount of trimming we derive new kernel type estimators. Extensive simulation suggests that one of the new considered kernels leads to a highly competitive estimator against virtually any other available alternative in this framework. Moreover we propose an adaptive selection method for the amount of top data used in estimation based on the trimming procedure minimizing the asymptotic mean squared error. We also provide an illustration of this approach to simulated as well as to real-world MTPL insurance data.

Model selection in latent block models has been a challenging but important task in the field of statistics. Specifically, a major challenge is encountered when constructing a test on a block structure obtained by applying a specific clustering algorithm to a finite size matrix. In this case, it becomes crucial to consider the selective bias in the block structure, that is, the block structure is selected from all the possible cluster memberships based on some criterion by the clustering algorithm. To cope with this problem, this study provides a selective inference method for latent block models. Specifically, we construct a statistical test on a set of row and column cluster memberships of a latent block model, which is given by a squared residue minimization algorithm. The proposed test, by its nature, includes and thus can also be used as the test on the set of row and column cluster numbers. We also propose an approximated version of the test based on simulated annealing to avoid combinatorial explosion in searching the optimal block structure. The results show that the proposed exact and approximated tests work effectively, compared to the naive test that did not take the selective bias into account.

We study the problem of density estimation on $[0,1]$ under ${\mathbb{L}}^p$ norm. We carry out a new piecewise polynomial estimator and prove that it is simultaneously (near)-minimax over a very wide range of Besov classes ${\mathcal{B}}_{\mathrm{\pi },\mathrm{\infty }}^{\mathit{\alpha }}(R)$. In particular, we may deal with unbounded densities and shed light on the minimax rates of convergence when $\mathrm{\pi }<p$ and $\mathit{\alpha }\in (1∕\mathrm{\pi }-1∕p,1∕\mathrm{\pi }]$.

This paper develops a unified framework for deriving optimal designs for hypothesis testing in the presence of several heteroscedastic groups. In particular, the obtained optimal designs are generalized Neyman allocations involving only two experimental groups. In order to account for the ordering among the treatments, particularly relevant in the clinical context for ethical reasons, we provide the optimal design for testing under constraints reflecting their effectiveness. The advantages of the suggested allocations are illustrated both theoretically and through several numerical examples, also compared with other designs proposed in the literature, showing a substantial gain in terms of both power and ethics.

In this paper, we focus on the high-dimensional location testing problem of directional data under the assumption of rotationally symmetric distributions, where the data dimension is potentially much larger than the sample size. We study the family of directional weighted spatial sign tests for this testing problem and establish the asymptotic null distributions and local power properties of this family. In particular, we find that the test based on the inverse norm weight, named as the inverse norm weight spatial sign test, has the maximum asymptotic power in this family. As demonstrated by extensive numerical results, the inverse norm weight spatial sign test has advantages in empirical power compared with some other members in the family as well as some existing tests.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moments, we show that iteratively reweighted ${\ell }_1$-penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as $\mathcal{O}(logs+loglogd)$ iterations to reach such an oracle estimator, where s and d denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

The class of observation-driven models (ODMs) includes many models of non-linear time series which, in a fashion similar to, yet different from, hidden Markov models (HMMs), involve hidden variables. Interestingly, in contrast to most HMMs, ODMs enjoy likelihoods that can be computed exactly with computational complexity of the same order as the number of observations, making maximum likelihood estimation the privileged approach for statistical inference for these models. A celebrated example of general order ODMs is the GARCH$(p,q)$ model, for which ergodicity and inference has been studied extensively. However little is known on more general models, in particular integer-valued ones, such as the log-linear Poisson GARCH or the NBIN-GARCH of order $(p,q)$ about which most of the existing results seem restricted to the case $p=q=1$. Here we fill this gap and derive ergodicity conditions for general ODMs. The consistency and the asymptotic normality of the maximum likelihood estimator (MLE) can then be derived using the method already developed for first order ODMs.

It is a standard assumption in the density deconvolution problem that the characteristic function of the measurement error distribution is non-zero on the real line. While this condition is assumed in the majority of existing works on the topic, there are many problem instances of interest where it is violated. In this paper we focus on non–standard settings where the characteristic function of the measurement errors has zeros, and study how zeros multiplicity affects the estimation accuracy. For a prototypical problem of this type we demonstrate that the best achievable estimation accuracy is determined by the multiplicity of zeros, the rate of decay of the error characteristic function, as well as by the smoothness and the tail behavior of the estimated density. We derive lower bounds on the minimax risk and develop optimal in the minimax sense estimators. In addition, we consider the problem of adaptive estimation and propose a data–driven estimator that automatically adapts to unknown smoothness and tail behavior of the density to be estimated.

This article aims to use beamforming, a covariate-assisted data projection method to solve the problem of variable selection for multivariate random-effects regression models. The new approach attempts to explore the covariance structure in the data with a small number of random-effects covariates. The basic premise behind the proposal is to scan through a covariate space with a series of forward filters named null-beamformers; each is tailored to a particular covariate in the space and resistant to interference effects originating from other covariates. Applying the proposed method to simulated and real multivariate regression data, we show that it can substantially outperform the existing methods of multivariate variable selection in terms of sensitivity and specificity. A theory on selection consistency is established under certain regularity conditions.

Shape restrictions such as monotonicity on functions often arise naturally in statistical modeling. We consider a Bayesian approach to the estimation of a monotone regression function and testing for monotonicity. We construct a prior distribution using piecewise constant functions. For estimation, a prior imposing monotonicity of the heights of these steps is sensible, but the resulting posterior is harder to analyze theoretically. We consider a “projection-posterior” approach, where a conjugate normal prior is used, but the monotonicity constraint is imposed on posterior samples by a projection map onto the space of monotone functions. We show that the resulting posterior contracts at the optimal rate $n^{-1∕3}$ under the ${\mathbb{L}}_1$-metric and at a nearly optimal rate under the empirical ${\mathbb{L}}_p$-metrics for $0<p\le 2$. The projection-posterior approach is also computationally more convenient. We also construct a Bayesian test for the hypothesis of monotonicity using the posterior probability of a shrinking neighborhood of the set of monotone functions. We show that the resulting test has a universal consistency property and obtain the separation rate which ensures that the resulting power function approaches one.

In this study, we consider a problem of monitoring parameter changes particularly in the presence of outliers. To propose a sequential procedure that is robust against outliers, we use the density power divergence to derive a detector and stopping time that make up our procedure. We first investigate the asymptotic properties of our sequential procedure for i.i.d. sequences and then extend the proposed procedure to stationary time series models, where we provide a set of sufficient conditions under which the proposed procedure has an asymptotically controlled size and consistency in power. As an application, our procedure is applied to the GARCH models. We demonstrate the validity and robustness of the proposed procedure through a simulation study. Finally, two real data analyses are provided to illustrate the usefulness of the proposed sequential procedure.

We study the non-parametric estimation of an unknown density f with support on ${\mathbb{R}}_{+}$ based on an i.i.d. sample with multiplicative measurement errors. The proposed fully-data driven procedure is based on the estimation of the Mellin transform of the density f, a regularisation of the inverse of the Mellin transform by a spectral cut-off and a data-driven model selection in order to deal with the upcoming bias-variance trade-off. We introduce and discuss further Mellin-Sobolev spaces which characterize the regularity of the unknown density f through the decay of its Mellin transform. Additionally, we show minimax-optimality over Mellin-Sobolev spaces of the data-driven density estimator and hence its adaptivity.

Statistical depths have been well studied for multivariate and functional data over the past few decades, but remain under-explored for point processes. A first attempt on the notion of point process depth was conducted recently where the depth was defined as a weighted product of two terms: (1) the probability of the number of events in each process and (2) the depth of the event times conditioned on the number of events by using a Mahalanobis depth. We point out that multivariate depths such as the Mahalanobis depth cannot be directly used because they often neglect the important ordering property in the point process events. To deal with this problem, we propose a model-based approach for point process systematically. In particular, we develop a Dirichlet-distribution-based framework on the conditional depth term, where the new methods are referred to as Dirichlet depths. We examine mathematical properties of the new depths and conduct asymptotic analysis. In addition, we illustrate the new methods using various simulated and real experiment data. It is found that the proposed framework provides a reasonable center-outward rank and the new methods have accurate decoding in one neural spike train dataset.

Split-Plot or Repeated Measures Designs with multiple groups occur naturally in sciences. Their analysis is usually based on the classical Repeated Measures ANOVA. Roughly speaking, the latter can be shown to be asymptotically valid for large sample sizes $n_i$ assuming a fixed number of groups a and time points d. However, for high-dimensional settings with $d>n_i$, this argument breaks down and statistical tests are often based on (standardized) quadratic forms. Furthermore, analysis of their limit behaviour is usually based on certain assumptions on how d converges to ∞ with respect to $n_i$. As this may be hard to argue in practice, we do not want to make such restrictions. Moreover, sometimes also the number of groups a may be large compared to d or $n_i$. To also have an impression about the behaviour of (standardized) quadratic forms as test statistic, we analyze their asymptotics under diverse settings on a, d and $n_i$. In fact, we combine all kinds of combinations, where they diverge or are bounded in a unified framework. To this aim, we assume equal covariance matrices between all groups. Studying the limit distributions in detail, we follow Sattler and Pauly (2018) and propose an approximation to obtain critical values. The resulting test and its approximation approach are investigated in an extensive simulation study focusing on the exceptional asymptotic frameworks that are the main focus of this work.