We consider the random-design least-squares regression problem within the reproducing kernel Hilbert space (RKHS) framework. Given a stream of independent and identically distributed input/output data, we aim to learn a regression function within an RKHS $\mathcal{H}$, even if the optimal predictor (i.e., the conditional expectation) is not in $\mathcal{H}$. In a stochastic approximation framework where the estimator is updated after each observation, we show that the averaged unregularized least-mean-square algorithm (a form of stochastic gradient descent), given a sufficient large step-size, attains optimal rates of convergence for a variety of regimes for the smoothnesses of the optimal prediction function and the functions in $\mathcal{H}$. Our results apply as well in the usual finite-dimensional setting of parametric least-squares regression, showing adaptivity of our estimator to the spectral decay of the covariance matrix of the covariates.

We consider a partially linear framework for modeling massive heterogeneous data. The major goal is to extract common features across all subpopulations while exploring heterogeneity of each subpopulation. In particular, we propose an aggregation type estimator for the commonality parameter that possesses the (nonasymptotic) minimax optimal bound and asymptotic distribution as if there were no heterogeneity. This oracle result holds when the number of subpopulations does not grow too fast. A plug-in estimator for the heterogeneity parameter is further constructed, and shown to possess the asymptotic distribution as if the commonality information were available. We also test the heterogeneity among a large number of subpopulations. All the above results require to regularize each subestimation as though it had the entire sample. Our general theory applies to the divide-and-conquer approach that is often used to deal with massive homogeneous data. A technical by-product of this paper is statistical inferences for general kernel ridge regression. Thorough numerical results are also provided to back up our theory.

In this paper, an over-sampled periodogram higher criticism (OPHC) test is proposed for the global detection of sparse periodic effects in a complex-valued time series. An explicit minimax detection boundary is established between the rareness and weakness of the complex sinusoids hidden in the series. The OPHC test is shown to be asymptotically powerful in the detectable region. Numerical simulations illustrate and verify the effectiveness of the proposed test. Furthermore, the periodogram over-sampled by $O(\log N)$ is proven universally optimal in global testing for periodicities under a mild minimum separation condition.

We extend the available asymptotic theory for autoregressive sieve estimators to cover the case of stationary and invertible linear processes driven by independent identically distributed (i.i.d.) infinite variance (IV) innovations. We show that the ordinary least squares sieve estimates, together with estimates of the impulse responses derived from these, obtained from an autoregression whose order is an increasing function of the sample size, are consistent and exhibit asymptotic properties analogous to those which obtain for a finite-order autoregressive process driven by i.i.d. IV errors. As these limit distributions cannot be directly employed for inference because they either may not exist or, where they do, depend on unknown parameters, a second contribution of the paper is to investigate the usefulness of bootstrap methods in this setting. Focusing on three sieve bootstraps: the wild and permutation bootstraps, and a hybrid of the two, we show that, in contrast to the case of finite variance innovations, the wild bootstrap requires an infeasible correction to be consistent, whereas the other two bootstrap schemes are shown to be consistent (the hybrid for symmetrically distributed innovations) under general conditions.

We consider the parameter estimation of Markov chain when the unknown transition matrix belongs to an exponential family of transition matrices. Then we show that the sample mean of the generator of the exponential family is an asymptotically efficient estimator. Further, we also define a curved exponential family of transition matrices. Using a transition matrix version of the Pythagorean theorem, we give an asymptotically efficient estimator for a curved exponential family.

This paper presents a unified geometric framework for the statistical analysis of a general ill-posed linear inverse model which includes as special cases noisy compressed sensing, sign vector recovery, trace regression, orthogonal matrix estimation and noisy matrix completion. We propose computationally feasible convex programs for statistical inference including estimation, confidence intervals and hypothesis testing. A theoretical framework is developed to characterize the local estimation rate of convergence and to provide statistical inference guarantees. Our results are built based on the local conic geometry and duality. The difficulty of statistical inference is captured by the geometric characterization of the local tangent cone through the Gaussian width and Sudakov estimate.

Let ${\mathbf{A}}_{p}=\frac{{\mathbf{Y}}{\mathbf{Y}}^{*}}{m}$ and ${\mathbf{B}}_{p}=\frac{{\mathbf{X}}{\mathbf{X}}^{*}}{n}$ be two independent random matrices where ${\mathbf{X}}=(X_{ij})_{p\times n}$ and ${\mathbf{Y}}=(Y_{ij})_{p\times m}$ respectively consist of real (or complex) independent random variables with $\mathbb{E}X_{ij}=\mathbb{E}Y_{ij}=0$, $\mathbb{E}|X_{ij}|^{2}=\mathbb{E}|Y_{ij}|^{2}=1$. Denote by $\lambda_{1}$ the largest root of the determinantal equation $\det(\lambda{\mathbf{A}}_{p}-{\mathbf{B}}_{p})=0$. We establish the Tracy–Widom type universality for $\lambda_{1}$ under some moment conditions on $X_{ij}$ and $Y_{ij}$ when $p/m$ and $p/n$ approach positive constants as $p\rightarrow\infty$.

We establish a Cramér-type moderate deviation result for self-normalized sums of weakly dependent random variables, where the moment requirement is much weaker than the non-self-normalized counterpart. The range of the moderate deviation is shown to depend on the moment condition and the degree of dependence of the underlying processes. We consider three types of self-normalization: the equal-block scheme, the big-block-small-block scheme and the interlacing scheme. Simulation study shows that the latter can have a better finite-sample performance. Our result is applied to multiple testing and construction of simultaneous confidence intervals for ultra-high dimensional time series mean vectors.

Serial correlation in the residuals of time series models can cause bias in both model estimation and prediction. However, models with such serially correlated residuals are difficult to estimate, especially when the regression function is nonlinear. Existing estimation methods require strong assumption for the relation between the residuals and the regressors, which excludes the commonly used autoregressive models in time series analysis. By extending the Whittle likelihood estimation, this paper investigates in details a semi-parametric autoregressive model with ARMA sequence of residuals. Asymptotic normality of the estimators is established, and a model selection procedure is proposed. Numerical examples are employed to illustrate the performance of the proposed estimation method and the necessity of incorporating the serial correlation in the residuals.

We consider the problem of testing multiple quantum hypotheses $\{\rho_{1}^{\otimes n},\ldots,\rho_{r}^{\otimes n}\}$, where an arbitrary prior distribution is given and each of the $r$ hypotheses is $n$ copies of a quantum state. It is known that the minimal average error probability $P_{e}$ decays exponentially to zero, that is, $P_{e}=\exp\{-\xi n+o(n)\}$. However, this error exponent $\xi$ is generally unknown, except for the case that $r=2$.

In this paper, we solve the long-standing open problem of identifying the above error exponent, by proving Nussbaum and Szkoła’s conjecture that $\xi=\min_{i\neq j}C(\rho_{i},\rho_{j})$. The right-hand side of this equality is called the multiple quantum Chernoff distance, and $C(\rho_{i},\rho_{j}):=\max_{0\leq s\leq1}\{-\log\operatorname{Tr}\rho_{i}^{s}\rho_{j}^{1-s}\}$ has been previously identified as the optimal error exponent for testing two hypotheses, $\rho_{i}^{\otimes n}$ versus $\rho_{j}^{\otimes n}$.

The main ingredient of our proof is a new upper bound for the average error probability, for testing an ensemble of finite-dimensional, but otherwise general, quantum states. This upper bound, up to a states-dependent factor, matches the multiple-state generalization of Nussbaum and Szkoła’s lower bound. Specialized to the case $r=2$, we give an alternative proof to the achievability of the binary-hypothesis Chernoff distance, which was originally proved by Audenaert et al.

A statistical functional, such as the mean or the median, is called elicitable if there is a scoring function or loss function such that the correct forecast of the functional is the unique minimizer of the expected score. Such scoring functions are called strictly consistent for the functional. The elicitability of a functional opens the possibility to compare competing forecasts and to rank them in terms of their realized scores. In this paper, we explore the notion of elicitability for multi-dimensional functionals and give both necessary and sufficient conditions for strictly consistent scoring functions. We cover the case of functionals with elicitable components, but we also show that one-dimensional functionals that are not elicitable can be a component of a higher order elicitable functional. In the case of the variance, this is a known result. However, an important result of this paper is that spectral risk measures with a spectral measure with finite support are jointly elicitable if one adds the “correct” quantiles. A direct consequence of applied interest is that the pair (Value at Risk, Expected Shortfall) is jointly elicitable under mild conditions that are usually fulfilled in risk management applications.

Functional covariates are common in many medical, biodemographic and neuroimaging studies. The aim of this paper is to study functional Cox models with right-censored data in the presence of both functional and scalar covariates. We study the asymptotic properties of the maximum partial likelihood estimator and establish the asymptotic normality and efficiency of the estimator of the finite-dimensional estimator. Under the framework of reproducing kernel Hilbert space, the estimator of the coefficient function for a functional covariate achieves the minimax optimal rate of convergence under a weighted $L_{2}$-risk. This optimal rate is determined jointly by the censoring scheme, the reproducing kernel and the covariance kernel of the functional covariates. Implementation of the estimation approach and the selection of the smoothing parameter are discussed in detail. The finite sample performance is illustrated by simulated examples and a real application.

In the present paper, we consider the application of overcomplete dictionaries to the solution of general ill-posed linear inverse problems. In the context of regression problems, there has been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso and its variants, is based on maximizing the likelihood, and relies on stringent assumptions on the dictionary, the so-called compatibility conditions, for a proof of its convergence rates. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction properties imposed by the linear operator.

In what follows, we bypass this difficulty by a novel approach, which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations, and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide sharp oracle inequalities for the risk in a non-asymptotic setting.

The performance of spectral clustering can be considerably improved via regularization, as demonstrated empirically in Amini et al. [Ann. Statist. 41 (2013) 2097–2122]. Here, we provide an attempt at quantifying this improvement through theoretical analysis. Under the stochastic block model (SBM), and its extensions, previous results on spectral clustering relied on the minimum degree of the graph being sufficiently large for its good performance. By examining the scenario where the regularization parameter $\tau$ is large, we show that the minimum degree assumption can potentially be removed. As a special case, for an SBM with two blocks, the results require the maximum degree to be large (grow faster than $\log n$) as opposed to the minimum degree. More importantly, we show the usefulness of regularization in situations where not all nodes belong to well-defined clusters. Our results rely on a ‘bias-variance’-like trade-off that arises from understanding the concentration of the sample Laplacian and the eigengap as a function of the regularization parameter. As a byproduct of our bounds, we propose a data-driven technique DKest (standing for estimated Davis–Kahan bounds) for choosing the regularization parameter. This technique is shown to work well through simulations and on a real data set.

In this paper, we deal with the problem of marginalization over and conditioning on two disjoint subsets of the node set of chain graphs (CGs) with the LWF Markov property. For this purpose, we define the class of chain mixed graphs (CMGs) with three types of edges and, for this class, provide a separation criterion under which the class of CMGs is stable under marginalization and conditioning and contains the class of LWF CGs as its subclass. We provide a method for generating such graphs after marginalization and conditioning for a given CMG or a given LWF CG. We then define and study the class of anterial graphs, which is also stable under marginalization and conditioning and contains LWF CGs, but has a simpler structure than CMGs.