By introducing suitable loss random variables of detection, we obtain optimal tests in terms of the stopping time or alarm time for Bayesian change-point detection not only for a general prior distribution of change-points but also for observations being a Markov process. Moreover, the optimal (minimal) average detection delay is proved to be equal to $1$ for any (possibly large) average run length to false alarm if the number of possible change-points is finite.

We study in this paper computational and statistical boundaries for submatrix localization. Given one observation of (one or multiple nonoverlapping) signal submatrix (of magnitude $\lambda$ and size $k_{m}\times k_{n}$) embedded in a large noise matrix (of size $m\times n$), the goal is to optimal identify the support of the signal submatrix computationally and statistically.

Two transition thresholds for the signal-to-noise ratio $\lambda/\sigma$ are established in terms of $m$, $n$, $k_{m}$ and $k_{n}$. The first threshold, $\sf SNR_{c}$, corresponds to the computational boundary. We introduce a new linear time spectral algorithm that identifies the submatrix with high probability when the signal strength is above the threshold $\sf SNR_{c}$. Below this threshold, it is shown that no polynomial time algorithm can succeed in identifying the submatrix, under the hidden clique hypothesis. The second threshold, $\sf SNR_{s}$, captures the statistical boundary, below which no method can succeed in localization with probability going to one in the minimax sense. The exhaustive search method successfully finds the submatrix above this threshold. In marked contrast to submatrix detection and sparse PCA, the results show an interesting phenomenon that $\sf SNR_{c}$ is always significantly larger than $\sf SNR_{s}$ under the sub-Gaussian error model, which implies an essential gap between statistical optimality and computational efficiency for submatrix localization.

The assumption of separability of the covariance operator for a random image or hypersurface can be of substantial use in applications, especially in situations where the accurate estimation of the full covariance structure is unfeasible, either for computational reasons, or due to a small sample size. However, inferential tools to verify this assumption are somewhat lacking in high-dimensional or functional data analysis settings, where this assumption is most relevant. We propose here to test separability by focusing on $K$-dimensional projections of the difference between the covariance operator and a nonparametric separable approximation. The subspace we project onto is one generated by the eigenfunctions of the covariance operator estimated under the separability hypothesis, negating the need to ever estimate the full nonseparable covariance. We show that the rescaled difference of the sample covariance operator with its separable approximation is asymptotically Gaussian. As a by-product of this result, we derive asymptotically pivotal tests under Gaussian assumptions, and propose bootstrap methods for approximating the distribution of the test statistics. We probe the finite sample performance through simulations studies, and present an application to log-spectrogram images from a phonetic linguistics dataset.

Many applications of block designs exhibit neighbor and edge effects. A popular remedy is to use the circular design coupled with the interference model. The search for optimal or efficient designs has been intensively studied in recent years. The circular neighbor balanced designs at distances 1 and 2 (CNBD2), including orthogonal array of type I ($\mathrm{OA}_{I}$) of strength $2$, are the two major designs proposed in literature for the purpose of estimating the direct treatment effects. They are shown to be optimal within some reasonable subclasses of designs. By using benchmark designs in approximate design theory, we show that CNBD2 is highly efficient among all possible designs when the error terms are homoscedastic and uncorrelated. However, when the error terms are correlated, these designs will be outperformed significantly by other designs. Note that CNBD2 fall into the special catalog of pseudo symmetric designs, and they only exist when the number of treatments is larger than the block size and the number of blocks is multiple of some constants. In this paper, we elaborate equivalent conditions for any design, pseudo symmetric or not, to be universally optimal for any size of experiment and any covariance structure of the error terms. This result is novel for circular designs and sheds light on other similar models in the search for optimal or efficient asymmetric designs.

Performance bounds are given for exploratory co-clustering/blockmodeling of bipartite graph data, where we assume the rows and columns of the data matrix are samples from an arbitrary population. This is equivalent to assuming that the data is generated from a nonsmooth graphon. It is shown that co-clusters found by any method can be extended to the row and column populations, or equivalently that the estimated blockmodel approximates a blocked version of the generative graphon, with estimation error bounded by $O_{P}(n^{-1/2})$. Analogous performance bounds are also given for degree-corrected blockmodels and random dot product graphs, with error rates depending on the dimensionality of the latent variable space.

The present paper considers the problem of estimating a linear functional $\Phi=\int_{-\infty}^{\infty}\varphi(x)f(x)\,dx$ of an unknown deconvolution density $f$ on the basis of $n$ i.i.d. observations, $Y_{1},\ldots,Y_{n}$ of $Y=\theta+\xi$, where $\xi$ has a known pdf $g$, and $f$ is the pdf of $\theta$. The objective of the present paper is to develop the a general minimax theory of estimating $\Phi$, and to relate this problem to estimation of functionals $\Phi_{n}=n^{-1}\sum_{i=1}^{n}\varphi(\theta_{i})$ in indirect observations. In particular, we offer a general, Fourier transform based approach to estimation of $\Phi$ (and $\Phi_{n}$) and derive upper and minimax lower bounds for the risk for an arbitrary square integrable function $\varphi$. Furthermore, using technique of inversion formulas, we extend the theory to a number of situations when the Fourier transform of $\varphi$ does not exist, but $\Phi$ can be presented as a functional of the Fourier transform of $f$ and its derivatives. The latter enables us to construct minimax estimators of the functionals that have never been handled before such as the odd absolute moments and the generalized moments of the deconvolution density. Finally, we generalize our results to the situation when the vector $\mathbf{{\theta}}$ is sparse and the objective is estimating $\Phi$ (or $\Phi_{n}$) over the nonzero components only. As a direct application of the proposed theory, we automatically recover multiple recent results and obtain a variety of new ones such as, for example, estimation of the mixing probability density function with classical and Berkson errors and estimation of the $(2M+1)$-th absolute moment of the deconvolution density.

In this work, we develop change-point methods for statistics of high-frequency data. The main interest is in the volatility of an Itô semimartingale, the latter being discretely observed over a fixed time horizon. We construct a minimax-optimal test to discriminate continuous paths from paths with volatility jumps, and it is shown that the test can be embedded into a more general theory to infer the smoothness of volatilities. In a high-frequency setting, we prove weak convergence of the test statistic under the hypothesis to an extreme value distribution. Moreover, we develop methods to infer changes in the Hurst parameters of fractional volatility processes. A simulation study is conducted to demonstrate the performance of our methods in finite-sample applications.

This paper presents a new and efficient method for the construction of optimal designs for regression models with dependent error processes. In contrast to most of the work in this field, which starts with a model for a finite number of observations and considers the asymptotic properties of estimators and designs as the sample size converges to infinity, our approach is based on a continuous time model. We use results from stochastic analysis to identify the best linear unbiased estimator (BLUE) in this model. Based on the BLUE, we construct an efficient linear estimator and corresponding optimal designs in the model for finite sample size by minimizing the mean squared error between the optimal solution in the continuous time model and its discrete approximation with respect to the weights (of the linear estimator) and the optimal design points, in particular in the multiparameter case.

In contrast to previous work on the subject, the resulting estimators and corresponding optimal designs are very efficient and easy to implement. This means that they are practically not distinguishable from the weighted least squares estimator and the corresponding optimal designs, which have to be found numerically by nonconvex discrete optimization. The advantages of the new approach are illustrated in several numerical examples.

In this paper, we consider the extreme behavior of the extremal eigenvalues of white Wishart matrices, which plays an important role in multivariate analysis. In particular, we focus on the case when the dimension of the feature $p$ is much larger than or comparable to the number of observations $n$, a common situation in modern data analysis. We provide asymptotic approximations and bounds for the tail probabilities of the extremal eigenvalues. Moreover, we construct efficient Monte Carlo simulation algorithms to compute the tail probabilities. Simulation results show that our method has the best performance among known approximation approaches, and furthermore provides an efficient and accurate way for evaluating the tail probabilities in practice.

To aid in the discrimination between two, possibly nonlinear, regression models, we study the construction of experimental designs. Considering that each of these two models might be only approximately specified, robust “maximin” designs are proposed. The rough idea is as follows. We impose neighbourhood structures on each regression response, to describe the uncertainty in the specifications of the true underlying models. We determine the least favourable—in terms of Kullback–Leibler divergence—members of these neighbourhoods. Optimal designs are those maximizing this minimum divergence. Sequential, adaptive approaches to this maximization are studied. Asymptotic optimality is established.

We consider nonparametric Bayesian inference in a reflected diffusion model $dX_{t}=b(X_{t})\,dt+\sigma(X_{t})\,dW_{t}$, with discretely sampled observations $X_{0},X_{\Delta},\ldots,X_{n\Delta}$. We analyse the nonlinear inverse problem corresponding to the “low frequency sampling” regime where $\Delta>0$ is fixed and $n\to\infty$. A general theorem is proved that gives conditions for prior distributions $\Pi$ on the diffusion coefficient $\sigma$ and the drift function $b$ that ensure minimax optimal contraction rates of the posterior distribution over Hölder–Sobolev smoothness classes. These conditions are verified for natural examples of nonparametric random wavelet series priors. For the proofs, we derive new concentration inequalities for empirical processes arising from discretely observed diffusions that are of independent interest.

Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires careful tuning of key parameters. Here, we introduce implicit stochastic gradient descent procedures, which involve parameter updates that are implicitly defined. Intuitively, implicit updates shrink standard stochastic gradient descent updates. The amount of shrinkage depends on the observed Fisher information matrix, which does not need to be explicitly computed; thus, implicit procedures increase stability without increasing the computational burden. Our theoretical analysis provides the first full characterization of the asymptotic behavior of both standard and implicit stochastic gradient descent-based estimators, including finite-sample error bounds. Importantly, analytical expressions for the variances of these stochastic gradient-based estimators reveal their exact loss of efficiency. We also develop new algorithms to compute implicit stochastic gradient descent-based estimators for generalized linear models, Cox proportional hazards, M-estimators, in practice, and perform extensive experiments. Our results suggest that implicit stochastic gradient descent procedures are poised to become a workhorse for approximate inference from large data sets.

For a joint model-based and design-based inference, we establish functional central limit theorems for the Horvitz–Thompson empirical process and the Hájek empirical process centered by their finite population mean as well as by their super-population mean in a survey sampling framework. The results apply to single-stage unequal probability sampling designs and essentially only require conditions on higher order correlations. We apply our main results to a Hadamard differentiable statistical functional and illustrate its limit behavior by means of a computer simulation.

In a very recent work, Basu and Owen [Found. Comput. Math. 17 (2017) 467–496] propose the use of scrambled geometric nets in numerical integration when the domain is a product of $s$ arbitrary spaces of dimension $d$ having a certain partitioning constraint. It was shown that for a class of smooth functions, the integral estimate has variance $O(n^{-1-2/d}(\log n)^{s-1})$ for scrambled geometric nets compared to $O(n^{-1})$ for ordinary Monte Carlo. The main idea of this paper is to expand on the work by Loh [Ann. Statist. 31 (2003) 1282–1324] to show that the scrambled geometric net estimate has an asymptotic normal distribution for certain smooth functions defined on products of suitable subsets of $\mathbb{R}^{d}$.

In this paper, we resolve a longstanding open statistical problem. The problem is to mathematically prove Yule’s 1926 empirical finding of “nonsense correlation” [J. Roy. Statist. Soc. 89 (1926) 1–63], which we do by analytically determining the second moment of the empirical correlation coefficient \begin{eqnarray*}&&\theta:=\frac{\int_{0}^{1}W_{1}(t)W_{2}(t)\,dt-\int_{0}^{1}W_{1}(t)\,dt\int_{0}^{1}W_{2}(t)\,dt}{\sqrt{\int_{0}^{1}W^{2}_{1}(t)\,dt-(\int_{0}^{1}W_{1}(t)\,dt)^{2}}\sqrt{\int_{0}^{1}W^{2}_{2}(t)\,dt-(\int_{0}^{1}W_{2}(t)\,dt)^{2}}},\end{eqnarray*} of two independent Wiener processes, $W_{1},W_{2}$. Using tools from Fredholm integral equation theory, we successfully calculate the second moment of $\theta$ to obtain a value for the standard deviation of $\theta$ of nearly 0.5. The “nonsense” correlation, which we call “volatile” correlation, is volatile in the sense that its distribution is heavily dispersed and is frequently large in absolute value. It is induced because each Wiener process is “self-correlated” in time. This is because a Wiener process is an integral of pure noise, and thus its values at different time points are correlated. In addition to providing an explicit formula for the second moment of $\theta$, we offer implicit formulas for higher moments of $\theta$.

Principal component analysis (PCA) is a widely used method for dimension reduction. In high-dimensional data, the “signal” eigenvalues corresponding to weak principal components (PCs) do not necessarily separate from the bulk of the “noise” eigenvalues. Therefore, popular tests based on the largest eigenvalue have little power to detect weak PCs. In the special case of the spiked model, certain tests asymptotically equivalent to linear spectral statistics (LSS)—averaging effects over all eigenvalues—were recently shown to achieve some power.

We consider a “local alternatives” model for the spectrum of covariance matrices that allows a general correlation structure. We develop new tests to detect PCs in this model. While the top eigenvalue contains little information, due to the strong correlations between the eigenvalues we can detect weak PCs by averaging over all eigenvalues using LSS. We show that it is possible to find the optimal LSS, by solving a certain integral equation. To solve this equation, we develop efficient algorithms that build on our recent method for computing the limit empirical spectrum [Dobriban (2015)]. The solvability of this equation also presents a new perspective on phase transitions in spiked models.